Dense Associative Memories and its Role in Machine Learning

First, we will discuss recent developments connecting Dense Associative Memories to the classical problem of clustering, and highlight how DenseAMs enable us to solve the discrete clustering problem with an end-to-end differentiable scheme. We will then demonstrate how DenseAMs also enable a novel differentiable simplified solution to problem of deep discrete clustering where we need to both perform discrete clustering while learning high-fidelity compressed representations. Finally, we will discuss the connection between DenseAMs and kernel machines, and demonstrate how ideas from kernel machines can be utilized to improve the size-expressivity trade-off in DenseAM networks.