Elasticity management of Streaming Data Analytics Flows on clouds

Alireza Khoshkbarforoushha; Alireza Khosravian; Rajiv Ranjan

doi:10.1016/j.jcss.2016.11.002

Elasticity management of Streaming Data Analytics Flows on clouds

Alireza Khoshkbarforoushha^*
, Alireza Khosravian
, Rajiv Ranjan

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

30 Citations (Scopus)

Abstract

In this paper, we present a framework for resource management of Streaming Data Analytics Flows (SDAF). Using advanced techniques in control and optimization theory, we design an adaptive control system tailored to the data ingestion, analytics, and storage layers of the SDAF that is able to continuously detect and self-adapt to workload changes for meeting the users’ service level objectives. Our experiments based on a real-world SDAF show that, the proposed control scheme is able to reduce the deviation from desired utilization of resources by up to 48% compared to existing techniques.

Original language	English
Pages (from-to)	24-40
Number of pages	17
Journal	Journal of Computer and System Sciences
Volume	89
DOIs	https://doi.org/10.1016/j.jcss.2016.11.002
Publication status	Published - Nov 2017

Access to Document

10.1016/j.jcss.2016.11.002

Cite this

@article{f3c2bbe2632f44a19835fa75ba836871,

title = "Elasticity management of Streaming Data Analytics Flows on clouds",

abstract = "In this paper, we present a framework for resource management of Streaming Data Analytics Flows (SDAF). Using advanced techniques in control and optimization theory, we design an adaptive control system tailored to the data ingestion, analytics, and storage layers of the SDAF that is able to continuously detect and self-adapt to workload changes for meeting the users{\textquoteright} service level objectives. Our experiments based on a real-world SDAF show that, the proposed control scheme is able to reduce the deviation from desired utilization of resources by up to 48\% compared to existing techniques.",

keywords = "Control theory, Data analytics flow, Data-intensive workloads, Multi-objective optimization, Public clouds, Resource management",

author = "Alireza Khoshkbarforoushha and Alireza Khosravian and Rajiv Ranjan",

note = "Publisher Copyright: {\textcopyright} 2016 Elsevier Inc.",

year = "2017",

month = nov,

doi = "10.1016/j.jcss.2016.11.002",

language = "English",

volume = "89",

pages = "24--40",

journal = "Journal of Computer and System Sciences",

issn = "0022-0000",

publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Elasticity management of Streaming Data Analytics Flows on clouds

AU - Khoshkbarforoushha, Alireza

AU - Khosravian, Alireza

AU - Ranjan, Rajiv

PY - 2017/11

Y1 - 2017/11

N2 - In this paper, we present a framework for resource management of Streaming Data Analytics Flows (SDAF). Using advanced techniques in control and optimization theory, we design an adaptive control system tailored to the data ingestion, analytics, and storage layers of the SDAF that is able to continuously detect and self-adapt to workload changes for meeting the users’ service level objectives. Our experiments based on a real-world SDAF show that, the proposed control scheme is able to reduce the deviation from desired utilization of resources by up to 48% compared to existing techniques.

AB - In this paper, we present a framework for resource management of Streaming Data Analytics Flows (SDAF). Using advanced techniques in control and optimization theory, we design an adaptive control system tailored to the data ingestion, analytics, and storage layers of the SDAF that is able to continuously detect and self-adapt to workload changes for meeting the users’ service level objectives. Our experiments based on a real-world SDAF show that, the proposed control scheme is able to reduce the deviation from desired utilization of resources by up to 48% compared to existing techniques.

KW - Control theory

KW - Data analytics flow

KW - Data-intensive workloads

KW - Multi-objective optimization

KW - Public clouds

KW - Resource management

UR - https://www.scopus.com/pages/publications/85007550368

U2 - 10.1016/j.jcss.2016.11.002

DO - 10.1016/j.jcss.2016.11.002

M3 - Article

SN - 0022-0000

VL - 89

SP - 24

EP - 40

JO - Journal of Computer and System Sciences

JF - Journal of Computer and System Sciences

ER -

Elasticity management of Streaming Data Analytics Flows on clouds

Abstract

Access to Document

Other files and links

Fingerprint

Cite this