|
|
|
Sensor Storage
|
|
|
|
|
|
|
|
###### Web interface
|
|
|
|
|
|
|
|
1. Go to http://MOWGLI_HOST:8282/#/default/scenarioSensorStorageClusterPost
|
|
|
|
2. Click on *Try Out*
|
|
|
|
3. Provide these parameter about details of your evaluation run:
|
|
|
|
```
|
|
|
|
#the following parameters will be used to create the result folder structure
|
|
|
|
scenarioType: SENSORSTORAGE
|
|
|
|
cloud: <OPENSTACK/EC2>
|
|
|
|
dbmsType: <CASSANDRA/COUCHBASE>
|
|
|
|
dbmsConfiguration: short name for the applied config, e.g. 3nodes-weakConsistency
|
|
|
|
|
|
|
|
#the follwoing parameters will be used to control the evaluation execution
|
|
|
|
runs: <1-n> the number of evaluation execution for the specified scenario
|
|
|
|
cleanUp: <true/false> specifies the deletion of each DBMS VM after an executed scenario (true is recommended, false only for debugging)
|
|
|
|
```
|
|
|
|
|
|
|
|
For the actual scenario specification you find in the examples folder the evaluation scenarios presented in the paper. Just copy the JSON into the body and replace the the `TODO` values of `idCloud, idImage, idHardware, idLocation` (for databaseDataComponent and databaseSeedComponent) with the values of the previously fetched VM template.
|
|
|
|
|
|
|
|
Hit the button **Execute** and grab a :coffee: :tea:
|
|
|
|
|
|
|
|
###### Shell
|
|
|
|
|
|
|
|
|
|
|
|
1. Use the following call, change the first 8 lines according to your cloud.
|
|
|
|
```
|
|
|
|
RESOURCETYPE=VM
|
|
|
|
IDCLOUD=1
|
|
|
|
IDIMAGE=97
|
|
|
|
IDHARDWARE=112
|
|
|
|
IDLOCATION=122
|
|
|
|
SCENARIO_TYPE=SENSORSTORAGE
|
|
|
|
CLOUD=OPENSTACK
|
|
|
|
DBMS_TYPE=CASSANDRA
|
|
|
|
DBMS_CONFIG_NAME=some_random_name
|
|
|
|
RUNS=1
|
|
|
|
CLEANUP=false
|
|
|
|
MOWGLI_IP=x.x.x.x
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
curl -X POST "http://$MOWGLI_IP:8282/v1/scenario/sensorStorageCluster?scenarioType=$SCENARIO_TYPE&cloud=$CLOUD&dbmsType=$DBMS_TYPE&dbmsConfiguration=$DBMS_CONFIG_NAME&runs=$RUNS&cleanUp=$CLEANUP" -H "accept: application/json" -H "Content-Type: application/json" -d "{ \"workloadNetwork\": { \"type\": \"PRIVATE\" }, \"dbmsNetwork\": { \"type\": \"PRIVATE\" }, \"dbmsCluster\": { \"type\": \"CASSANDRA\", \"databaseDataComponent\": [ { \"instances\": 2, \"resource\": { \"resourceType\": \"$RESOURCETYPE\", \"idCloud\": $IDCLOUD, \"idImage\": $IDIMAGE, \"idHardware\": $IDHARDWARE, \"idLocation\": $IDLOCATION }, \"customConfiguration\": [ ], \"name\": \"DATA\", \"nodeConfiguration\": { \"dataMemory\": { \"envName\": \"DATAMEMORY\", \"envValue\": 0 }, \"indexMemory\": { \"envName\": \"INDEXMEMORY\", \"envValue\": 0 } } } ], \"databaseSeedComponent\": [ { \"instances\": 1, \"resource\": { \"resourceType\": \"$RESOURCETYPE\", \"idCloud\": $IDCLOUD, \"idImage\": $IDIMAGE, \"idHardware\": $IDHARDWARE, \"idLocation\": $IDLOCATION }, \"customConfiguration\": [ ], \"name\": \"SEED\", \"replicationFactor\": { \"envName\": \"REPLICATIONFACTOR\", \"envValue\": 3 }, \"nodeConfiguration\": { \"dataMemory\": { \"envName\": \"DATAMEMORY\", \"envValue\": 1500 }, \"indexMemory\": { \"envName\": \"INDEXMEMORY\", \"envValue\": 500 } } } ], \"databaseManagementComponent\": [ ] }, \"workload\": { \"dbEndpoints\": [ ], \"measurementConfig\": { \"interval\": 10, \"measurementType\": \"NONE\" }, \"workloadConfig\": { \"workloadType\": \"LOAD\", \"workloadClass\": \"com.yahoo.ycsb.workloads.CoreWorkload\", \"maxExecutionTime\": 1800, \"threadCount\": 16, \"recordCount\": 4000000, \"insertStart\": 0, \"insertCount\": 0, \"operations\": 1000, \"targetOps\": 0, \"fieldCount\": 10, \"fieldLength\": 500, \"readAllFileds\": true, \"readModifyWriteProportion\": 0, \"requestdistribution\": \"UNIFORM\", \"scanLengthDistribution\": \"UNIFORM\", \"insertOrder\": \"ORDERED\", \"readProportion\": 0.0, \"updateProportion\": 0.0, \"insertProportion\": 1.0, \"scanProportion\": 0, \"maxScanLength\": 1000, \"coreWorkloadInsertionRetryLimit\": 3, \"coreWorkloadInsertionRetryInterval\": 3 }, \"databaseConfig\": { \"databaseBinding\": \"CASSANDRA2\", \"endpointParameterName\": \"hosts\", \"tableParameterName\": \"cassandra.keyspace\", \"tableName\": \"ycsb\", \"configPorperties\": [ { \"name\": \"cassandra.writeconsistencylevel\", \"value\": \"ONE\" } ] } }}"
|
|
|
|
2. The output signalizes the successful run (after several minutes):
|
|
|
|
```
|
|
|
|
{
|
|
|
|
"code": 4,
|
|
|
|
"type": "ok",
|
|
|
|
"message": "Sensor Storage Cluster evaluation finished!"
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
### After evaluation start
|
|
|
|
|
|
|
|
Now the execution of the evaluation is started and you will see the VMs coming up in your cloud dashboard. In addition you can follow the logs of the evaluation via portainer: login via http://MOWGLI_HOST:9001/#/auth with the default credentials user: `admin` and password `mowgli19`
|
|
|
|
|
|
|
|
Check the logs of the evaluation-orchestrator container and follow the progress of the evaluation.
|
|
|
|
|
|
|
|
You can also check the resource consumption via the Chronograf dashboard (if port 8888 is exposed):
|
|
|
|
|
|
|
|
http://MOWGLI_HOST:8888
|
|
|
|
|
|
|
|
After a successful execution you will find the results under:
|
|
|
|
|
|
|
|
```
|
|
|
|
/opt/evaluation-results/sensorstorage/CLOUD/DBMS/CONFIGURATION
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Feel free to run multiple evaluation scenarios (in case you have enough EC2 credits :smirk: ) and play with the following template options:
|
|
|
|
|
|
|
|
```
|
|
|
|
instances: X (only modify the number of databaseDataComponent!)
|
|
|
|
replicationfactor: 1-numberOfNodes
|
|
|
|
maxExecutionTime: 60-3600 (unit seconds)
|
|
|
|
recordCount: 10000 - n (make sure your VM have enough disk space, 4000000 require around 60GB of disk per Cassandra node)
|
|
|
|
fieldCount: 10 (each field is 500 bytes)TODO: explain the mowgli-default security group in openstack which might need some manual adjustments
|
|
|
|
```
|
|
|
|
|
|
|
|
YCSB Multi Phase
|
|
|
|
|
|
|
|
Plotting |
|
|
\ No newline at end of file |
