Conference paper
Workshop paper
Dense Associative Memory with Epanechnikov energy
Abstract
We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel and enables exact memory retrieval with exponential capacity without requiring exponential separation functions. Uniquely, it introduces abundant additional emergent local minima while preserving perfect pattern recovery--a characteristic previously unseen in DenseAM literature. Empirical results show LSR generates significantly more local minima and produces samples with higher log-likelihood than LSE-based models, making it promising for both memory storage and generative tasks.
Related
Conference paper
Do not have enough data? Deep learning to the rescue!
Conference paper