Causal Markov Blanket Representation Learning for Domain Generalization Prediction

Abstract

The pursuit of generalizable representations remains to be a dynamic field in the realm of machine learning and computer vision. Existing methods aim to secure invariant representations by either harnessing domain expertise or leveraging data from multiple domains. In this paper, we propose a novel approach that identifies the Causal Markov Blanket (CMB) representations and improves the Out-of-distribution prediction performance. To do so, we first introduce a structural causal model (SCM) to describe the underlying causal mechanisms governing the data generation process and distribution shifts. We theoretically prove that under our proposed SCM, the predictive mechanism using CMB representations remains invariant and predictive across domains. Subsequently, we propose a three-phase CMB representation learning procedure conforming to the proposed SCM assumptions. In comparison to state-of-the-art domain generalization methods, our approach exhibits robustness and adaptability under distribution shifts.