Investigating the dynamics of spontaneous activity in energy-based neural networks

Generative models are machine learning models capable of producing data samples according to the properties of the training distribution. This feature makes them particularly interesting from a computational neuroscience perspective, since it allows to investigate the functional role of spontaneous brain activity. A widely studied class of generative models is that of Restricted Boltzmann Machines (RBM), which are used as building blocks for unsupervised deep learning architectures. Despite their relevance, the generative capabilities of RBMs have not been systematically studied. In this work we propose a method to explore the generation of data samples from RBMs trained on a classic dataset (MNIST) composed of handwritten digits. Our approach exploits the gradual tuning of a temperature parameter to investigate whether the model dynamics could be driven into meta-stable states, which would allow to effectively generate heterogeneous data samples by visiting nearby attractor basins. Even if the proposed method indeed allows to generate different digits starting from a certain (and possibly biased) network state, it is not capable of producing samples belonging to all digit classes in one single generation round.