TY - JOUR
T1 - A generic pixel-to-point comparison for simulated large-scale ecosystem properties and ground-based observations
T2 - An example from the Amazon region
AU - Rammig, Anja
AU - Heinke, Jens
AU - Hofhansl, Florian
AU - Verbeeck, Hans
AU - Baker, Timothy R.
AU - Christoffersen, Bradley
AU - Ciais, Philippe
AU - De Deurwaerder, Hannes
AU - Fleischer, Katrin
AU - Galbraith, David
AU - Guimberteau, Matthieu
AU - Huth, Andreas
AU - Johnson, Michelle
AU - Krujit, Bart
AU - Langerwisch, Fanny
AU - Meir, Patrick
AU - Papastefanou, Phillip
AU - Sampaio, Gilvan
AU - Thonicke, Kirsten
AU - Von Randow, Celso
AU - Zang, Christian
AU - Rödig, Edna
N1 - Publisher Copyright:
© 2017 Copernicus GmbH.All right reserved.
PY - 2018/12/21
Y1 - 2018/12/21
N2 - Comparing model output and observed data is an important step for assessing model performance and quality of simulation results. However, such comparisons are often hampered by differences in spatial scales between local point observations and large-scale simulations of grid cells or pixels. In this study, we propose a generic approach for a pixel-to-point comparison and provide statistical measures accounting for the uncertainty resulting from landscape variability and measurement errors in ecosystem variables. The basic concept of our approach is to determine the statistical properties of small-scale (within-pixel) variability and observational errors, and to use this information to correct for their effect when large-scale area averages (pixel) are compared to small-scale point estimates. We demonstrate our approach by comparing simulated values of aboveground biomass, woody productivity (woody net primary productivity, NPP) and residence time of woody biomass from four dynamic global vegetation models (DGVMs) with measured inventory data from permanent plots in the Amazon rainforest, a region with the typical problem of low data availability, potential scale mismatch and thus high model uncertainty. We find that the DGVMs under- and overestimate aboveground biomass by 25% and up to 60%, respectively. Our comparison metrics provide a quantitative measure for model-data agreement and show moderate to good agreement with the region-wide spatial biomass pattern detected by plot observations. However, all four DGVMs overestimate woody productivity and underestimate residence time of woody biomass even when accounting for the large uncertainty range of the observational data. This is because DGVMs do not represent the relation between productivity and residence time of woody biomass correctly. Thus, the DGVMs may simulate the correct large-scale patterns of biomass but for the wrong reasons. We conclude that more information about the underlying processes driving biomass distribution are necessary to improve DGVMs. Our approach provides robust statistical measures for any pixel-to-point comparison, which is applicable for evaluation of models and remote-sensing products.
AB - Comparing model output and observed data is an important step for assessing model performance and quality of simulation results. However, such comparisons are often hampered by differences in spatial scales between local point observations and large-scale simulations of grid cells or pixels. In this study, we propose a generic approach for a pixel-to-point comparison and provide statistical measures accounting for the uncertainty resulting from landscape variability and measurement errors in ecosystem variables. The basic concept of our approach is to determine the statistical properties of small-scale (within-pixel) variability and observational errors, and to use this information to correct for their effect when large-scale area averages (pixel) are compared to small-scale point estimates. We demonstrate our approach by comparing simulated values of aboveground biomass, woody productivity (woody net primary productivity, NPP) and residence time of woody biomass from four dynamic global vegetation models (DGVMs) with measured inventory data from permanent plots in the Amazon rainforest, a region with the typical problem of low data availability, potential scale mismatch and thus high model uncertainty. We find that the DGVMs under- and overestimate aboveground biomass by 25% and up to 60%, respectively. Our comparison metrics provide a quantitative measure for model-data agreement and show moderate to good agreement with the region-wide spatial biomass pattern detected by plot observations. However, all four DGVMs overestimate woody productivity and underestimate residence time of woody biomass even when accounting for the large uncertainty range of the observational data. This is because DGVMs do not represent the relation between productivity and residence time of woody biomass correctly. Thus, the DGVMs may simulate the correct large-scale patterns of biomass but for the wrong reasons. We conclude that more information about the underlying processes driving biomass distribution are necessary to improve DGVMs. Our approach provides robust statistical measures for any pixel-to-point comparison, which is applicable for evaluation of models and remote-sensing products.
UR - http://www.scopus.com/inward/record.url?scp=85059409634&partnerID=8YFLogxK
U2 - 10.5194/gmd-11-5203-2018
DO - 10.5194/gmd-11-5203-2018
M3 - Article
SN - 1991-959X
VL - 11
SP - 5203
EP - 5215
JO - Geoscientific Model Development
JF - Geoscientific Model Development
IS - 12
ER -