Identifying Selection Bias from Observational Data

Abstract. Access to a representative sample from the population is an assumption that underpins all of machine learning. Selection effects can cause observations to instead come from a subpopulation, by which our inferences may be subject to bias. It is therefore important to know whether or not a sample is affected by selection effects. We study under which conditions we can identify selection bias and give results for both parametric and non-parametric families of distributions. Based on these results we develop two practical methods to determine whether or not an observed sample comes from a distribution subject to selection bias. Through extensive evaluation on synthetic and real world data we verify that our methods beat the state of the art both in detecting as well as characterizing selection bias.

Implementation

the Python source code (March 2024) by David Kaltenpoth.

Related Publications

Kaltenpoth, D & Vreeken, J Identifying Selection Bias from Observational Data. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 8177-8185, AAAI, 2023. (oral presentation, 10.8% acceptance rate; 19.6% overall)