Commit edb64da0 authored by Daniel Seybold's avatar Daniel Seybold
Browse files

updated data analytics script

parent 9e2f48a1
......@@ -11,16 +11,14 @@ The following publications and data sets present the scientific concepts and res
## Publications
```
@INPROCEEDINGS{8968865,
author={D. {Seybold} and S. {Volpert} and S. {Wesner} and A. {Bauer} and N. {Herbst} and J. {Domaschka}},
booktitle={2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)},
title={Kaa: Evaluating Elasticity of Cloud-Hosted DBMS},
year={2019},
@INPROCEEDINGS{grohmann2020baloo,
author={Grohmann, Johannes and Seybold, Daniel and Eismann, Simon and Leznik, Mark and Samuel Kounev and Domaschka, Jörg},
booktitle={2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)},
title={Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS},
year={2020},
volume={},
number={},
pages={54-61},
url = {https://doi.org/10.1109/CloudCom.2019.00020}
}
number={}
}
```
......@@ -46,6 +44,21 @@ series = {SAC '20}
```
@INPROCEEDINGS{8968865,
author={D. {Seybold} and S. {Volpert} and S. {Wesner} and A. {Bauer} and N. {Herbst} and J. {Domaschka}},
booktitle={2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)},
title={Kaa: Evaluating Elasticity of Cloud-Hosted DBMS},
year={2019},
volume={},
number={},
pages={54-61},
url = {https://doi.org/10.1109/CloudCom.2019.00020}
}
```
```
@inproceedings{Seybold:2019:MFY:3297663.3310303,
author = {Seybold, Daniel and Keppler, Moritz and Gr\"{u}ndler, Daniel and Domaschka, J\"{o}rg},
......
......@@ -36,7 +36,7 @@ The IP of the host of the Mowgli framework is denoted in the following as *MOWGL
| property | required |
| ------------------------ | ------------------------------------------------------------ |
| image / operating system | Ubuntu 18.04 (recommended), any OS with docker and docker-compose should work |
| image / operating system | Ubuntu 18.04 (recommended), any Linux-based OS with docker and docker-compose installed will work |
| CPU cores | 4 |
| memory | 4GB |
| disk | 20GB |
......
# Usage
# Data Analytics
The automated data processing capabilities of Mowgli target benchmark and systems metric on a single evaluation run. In order to apply advanced data analytic capabilities, Mowgli provides a set of data analytic tools for objective-specific data postprocessing.
These tools are based on **python3** and briefly explained in the following sections.
## Run Evaluation
All data analytic tools are compatible with the data format of Mowgli 0.2. Exemplary data sets can be found under [data sets.](../publications/README.md)
After the successful registration of the cloud, Cloudiator will collect the cloud resource offerings of the cloud provider.
## Performance
Depending on the number of cloud resource offers this might take some minutes, so time for a coffee :coffee:
The [Baloo framework](../publications/README.md) provides an extension to Mowgli to predict the performance of cloud-hosted DBMS based on existing Mowgli data sets.
Mowgli will compose the cloud resource offerings into VM templates that are required by each evaluation scenario.
The respective data analytics code can be found on [CodeOcean](https://doi.org/10.24433/CO.6929232.v2) with exemplary data and an executable environment.
Now you can query the Mowgli framework for appropriate VM templates as described [here](Get-VM-Templates.md)
### Start evaluation
After getting the VM templates you are ready to start the evaluations :cloud: :hourglass: :trophy:
## Scalability
Mowgli supports four types of evaluation scenarios:
A supportive python script to visualize the throughput and latency of different evaluations scenarios and multiple evaluation runs of each scenario is provided in the [analytics](https://omi-gitlab.e-technik.uni-ulm.de/mowgli/evaluation-orchestrator/-/tree/master/analytics) folder of the evaluation orchestrator.
[Performance](Performance-Evaluation.md)
#### Configuration
[Scalability](Scalability-Evaluation.md)
The target evaluation results folders need to be configured directly in the `pandasBarplot.py` script, see examples starting from `line 23`
[Elasticity](Elasticity-Evaluation.md)
#### Execution
[Availability](Availability-Evaluation.md)
```
./python3 pandasBarplot.py
```
Please check the respective scenario pages for further details about the execution, the supported DBMS and workloads.
#### Example Result
![](../misc/scalability_throughput_example.png)
## Elasticity
## Evaluation Results
All evaluation results are stored to the file system of the host that runs the Mowgli Framework in the following structure
```
opt
|_evaluation-results
|_SCENARIO
|_CLOUD
|_DBMS
|_CONFIG
|_RUN_X
|_data # contains raw evaluation data
|_monitoring # contains system usage plots
|_specs # contains the applied templates
|_taskLogs # additional logs
|_timeseries # throughput plot of the evaluation run
|_plots # contains aggregated evaluation data over all runs (manual processing steps required)
```
#### Example Results
![](../misc/elasticity_single_config.png)
![](../misc/elasticity_multiple_configs.png)
## Availability
adanved plots
data frames
#### Example Result
![](../misc/availability_phases_analytics.png)
## Multi Criteria Decision Making
More details can be found in the respective sections.
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment