TY - JOUR
T1 - Towards building a data-intensive index for big data computing - A case study of Remote Sensing data processing
AU - Ma, Yan
AU - Wang, Lizhe
AU - Liu, Peng
AU - Ranjan, Rajiv
N1 - Publisher Copyright:
© 2015 Elsevier Inc. All rights reserved.
PY - 2015/10/20
Y1 - 2015/10/20
N2 - With the recent advances in Remote Sensing (RS) techniques, continuous Earth Observation is generating tremendous volume of RS data. The proliferation of RS data is revolutionizing the way in which RS data are processed and understood. Data with higher dimensionality, as well as the increasing requirement for real-time processing capabilities, have also given rise to the challenging issue of "Data-Intensive (DI) Computing". However, how to properly identify and qualify the DI issue remains a significant problem that is worth exploring. DI computing is a complex issue. While the huge data volume may be one of the reasons for this, some other factors could also be important. In this paper, we propose an empirical model (DIRS) of DI index to estimate RS applications. DIRS here is a novel empirical model (DIRS) that could quantify the DI issues in RS data processing with a normalized DI index. Through experimental analysis of the typical algorithms across the whole RS data processing flow, we identify the key factors that affect the DI issues mostly. Finally, combined with the empirical knowledge of domain experts, we formulate DIRS model to describe the correlations between the key factors and DI index. By virtue of experimental validation on more selected RS applications, we have found that DIRS model is an easy but promising approach.
AB - With the recent advances in Remote Sensing (RS) techniques, continuous Earth Observation is generating tremendous volume of RS data. The proliferation of RS data is revolutionizing the way in which RS data are processed and understood. Data with higher dimensionality, as well as the increasing requirement for real-time processing capabilities, have also given rise to the challenging issue of "Data-Intensive (DI) Computing". However, how to properly identify and qualify the DI issue remains a significant problem that is worth exploring. DI computing is a complex issue. While the huge data volume may be one of the reasons for this, some other factors could also be important. In this paper, we propose an empirical model (DIRS) of DI index to estimate RS applications. DIRS here is a novel empirical model (DIRS) that could quantify the DI issues in RS data processing with a normalized DI index. Through experimental analysis of the typical algorithms across the whole RS data processing flow, we identify the key factors that affect the DI issues mostly. Finally, combined with the empirical knowledge of domain experts, we formulate DIRS model to describe the correlations between the key factors and DI index. By virtue of experimental validation on more selected RS applications, we have found that DIRS model is an easy but promising approach.
KW - Big data
KW - Data-intensive computing
KW - Parallel computing
KW - Remote Sensing data processing
UR - http://www.scopus.com/inward/record.url?scp=84930826671&partnerID=8YFLogxK
U2 - 10.1016/j.ins.2014.10.006
DO - 10.1016/j.ins.2014.10.006
M3 - Article
SN - 0020-0255
VL - 319
SP - 171
EP - 188
JO - Information Sciences
JF - Information Sciences
ER -