Wasserstein barycenter model ensembling
Pierre Dognin, Igor Melnyk, et al.
ICLR 2019
Deep neural networks work well at approximating complicated functions when provided with data and trained by gradient descent methods. At the same time, there is a vast amount of existing functions that programmatically solve different tasks in a precise manner eliminating the need for training. In many cases, it is possible to decompose a task to a series of functions, of which for some we may prefer to use a neural network to learn the functionality, while for others the preferred method would be to use existing black-box functions. We propose a method for end-to-end training of a base neural network that integrates calls to existing black-box functions. We do so by approximating the black-box functionality with a differentiable neural network in a way that drives the base network to comply with the black-box function interface during the end-to-end optimization process. At inference time, we replace the differentiable estimator with its external black-box non-differentiable counterpart such that the base network output matches the input arguments of the black-box function. Using this “Estimate and Replace” paradigm, we train a neural network, end to end, to compute the input to black-box functionality while eliminating the need for intermediate labels. We show that by leveraging the existing precise black-box function during inference, the integrated model generalizes better than a fully differentiable model, and learns more efficiently compared to RL-based methods.
Pierre Dognin, Igor Melnyk, et al.
ICLR 2019
Alon Jacovi, Ori Bar El, et al.
KDD-Converse 2020
Paula Ta-Shma, Adnan Akbar, et al.
IEEE IoT Journal
Yossi Adi, Einat Kermany, et al.
ICLR 2017