Property-based Sensitivity Analysis: An approach to identify model implementation and integration errors

Diagnostic testing is an oft-recommended use of sensitivity analysis to assess correctness or plausibility of model behavior. In this paper we demonstrate the use of sensitivity analysis as a complementary first-pass software test for the validation of model behavior. Typical testing processes rely on comparing model outputs to results known to be correct. Such approaches are limited to specific model configurations and require that correct results be known in advance. Property-based Sensitivity Analysis (PbSA) examines model properties in terms of the behavior of parameter sensitivities to provide a line of evidence that the expected conceptual relationships between model factors and their effects are present. Unanticipated results can indicate issues to be corrected. The PbSA approach is also scalable as it can complement existing testing practices and be applied in conjunction with global sensitivity methods that can reuse existing model evaluations or are otherwise independent of the sampling scheme.