TY - JOUR
T1 - Applying Design of Experiments (DOE) to performance evaluation of commercial cloud services
AU - Li, Zheng
AU - OàBrien, Liam
AU - Zhang, He
AU - Ranjan, Rajiv
PY - 2013
Y1 - 2013
N2 - Appropriate performance evaluations of commercial Cloud services are crucial and beneficial for both customers and providers to understand the service runtime, while suitable experimental design and analysis would be vital for practical evaluation implementations. However, there seems to be a lack of effective methods for Cloud services performance evaluation. For example, in most of the existing evaluation studies, experimental factors (also called parameters or variables) were considered randomly and intuitively, experimental sample sizes were determined on the fly, and few experimental results were comprehensively analyzed. To address these issues, the authors suggest applying Design of Experiments (DOE) to Cloud services evaluation. To facilitate applying DOE techniques, this paper introduces an experimental factor framework and a set of DOE application scenarios. As such, new evaluators can explore and conveniently adapt our work to their own experiments for performance evaluation of commercial Cloud services.
AB - Appropriate performance evaluations of commercial Cloud services are crucial and beneficial for both customers and providers to understand the service runtime, while suitable experimental design and analysis would be vital for practical evaluation implementations. However, there seems to be a lack of effective methods for Cloud services performance evaluation. For example, in most of the existing evaluation studies, experimental factors (also called parameters or variables) were considered randomly and intuitively, experimental sample sizes were determined on the fly, and few experimental results were comprehensively analyzed. To address these issues, the authors suggest applying Design of Experiments (DOE) to Cloud services evaluation. To facilitate applying DOE techniques, this paper introduces an experimental factor framework and a set of DOE application scenarios. As such, new evaluators can explore and conveniently adapt our work to their own experiments for performance evaluation of commercial Cloud services.
KW - Cloud computing
KW - Commercial cloud services
KW - Experimental design
KW - Factor framework
KW - Performance evaluation
UR - http://www.scopus.com/inward/record.url?scp=84902975015&partnerID=8YFLogxK
U2 - 10.4018/jghpc.2013070107
DO - 10.4018/jghpc.2013070107
M3 - Article
SN - 1938-0259
VL - 5
SP - 75
EP - 93
JO - International Journal of Grid and High Performance Computing
JF - International Journal of Grid and High Performance Computing
IS - 3
ER -