Linking losses for density ratio and class-probability estimation

Given samples from two densities p and q, density ratio estimation (DRE) is the problem of estimating the ratio p/q. In this paper, we formally relate DRE and class-probability estimation (CPE), and theoretically justify the use of existing losses from one problem for the other. In the CPE to DRE direction, we show that essentially any CPE loss (e.g. logistic, exponential) minimises a Bregman divergence to the true density ratio, and thus can be used for DRE. We also show how different losses focus on accurately modelling different ranges of the density ratio, and use this to design new CPE losses for DRE. In the DRE to CPE direction, we argue that the least squares importance fitting method has potential use for bipartite ranking of instances with maximal accuracy at the head of the ranking. Our analysis relies on a novel Bregman divergence identity that may be of independent interest.