Predictor performance with stratified data and imbalanced classes

The disagreement between Vihinen1 and Kumar et al.2 over the presentation of EvoD3 raises important issues relevant to any binary classifier, including the problems of class imbalance, what constitutes an appropriate performance metric and the legitimacy of training on stratified data. Researchers need to be aware that competing strategies to calibrate and evaluate a classifier may lead to differing perceptions of performance.