The Hopfield model: study of the basins of attractions and the matrix factorization problem

Statistical mechanics is the starting point from which some of well known Machine Learning architectures are built. This is the case of the Hopfield Model, which has very strong relationships with Spin Glass theory and the Ising model. First introduced at the end of '70, the Hopfield Model is often addressed to as an associative memory since it has the ability to store patterns by simulating the activity of the human brain. In this base model, the key ingredient for the storage is a coupling matrix that is computed following the Hebb's rule. The storage capacity of the model strongly depends on how the patterns are stored, that is the way we associate an energy to each pattern and on the pattern size. Although many works related to Hopfield can be found in the scientific literature, some important open questions remain. In this dissertation we provide numerical results through Monte Carlo algorithms for the characterization of the basins of attraction of the model by computing the so called critical noise, namely the amount of information that each pattern can lose in order to let the model be able to retrieve it correctly. This analysis is repeated also for the Modern Hopfield model (Dense Associative memories), where the storage capacity is exponential. Recently, many researchers are focusing their efforts also in the ``inverse'' problem, where we are provided only with the coupling matrix and the goal is to infer the stored patterns. Up to now there is no algorithm able to solve this problem and we show that the retrieval of just one of the global minimum of the system starting by a random configuration is computationally hard, even if the stored patterns are very few.