Hypothesis testing for management: Evolving and answering closed questions using multiobjective visualization

In order to use models to understand deeply uncertain future conditions, managers must be able to pose and test hypotheses about their management problems. In Iterative Closed Question Methodology (ICQM), a series of closed questions are used to structure thinking about hypotheses while looking beyond a problem's existing modeling representation. Our research is exploring how ICQM can contribute to a framework called Many Objective Robust Decision Making (MORDM), which uses multiobjective optimization and ensembles of uncertain future states of the world to create and evaluate robust solutions for environmental management. A visualization software tool; AeroVis, has greatly aided implementation of MORDM, allowing a user to plot tradeoffs between conflicting objectives, "brush" their preferences on plotted and unplotted variables, and view visualizations of solution robustness. This visualization approach provides a rich set of conclusions which is not always well understood (i.e. the user can interpret results that the modeler did not intend). In this presentation, we explore how visualization tools iteratively generate and evaluate management hypotheses and conclusions. We discuss the types of conclusions that can be made from AeroVis MORDM visualizations and walk through experimental examples of how individuals reason with the decision support tool. This illustrates that working within an MORDM framework helps the user consider alternate model assumptions about future inputs, parameters and model structure, supporting the idea that model assumptions can provide useful scenarios for environmental management.