Bias learning in neural networks

Thanks to their exceptional ability to learn complex tasks, in recent years Artificial Neural Networks (ANNs) have revolutionized a wide range of fields. Their black-box nature, however, has produced concerns on the front of interpretability and biological plausibility, particularly in fields like neuroscience where knowledge of learning mechanisms is essential. This thesis investigates bias-only learning, a different training paradigm that fixes weights and restricts learning to be implemented via bias parameters. We examine the expressivity and task generalization abilities of networks trained in this constrained fashion, motivated by recent neuroscientific findings indicating that bias-like mechanisms may be essential for quick task adaptation. We analyze Single Layer Perceptrons (SLPs), Multi Layer Perceptrons (MLPs), and Recurrent Neural Networks (RNNs), both under full and bias-only training, across a diverse set of tasks including binary classifications of points in a plane, of MNIST-like datasets, and cognitively inspired tasks commonly used in neuroscience. In addition, we apply Explainable AI (XAI) techniques to better understand the internal representations and mechanisms underlying multi-task learning and catastrophic forgetting. Our findings demonstrate that, despite the limited parameter set, bias-trained networks exhibit surprisingly strong performance and biological plausibility, making them a promising direction for both AI scalability and neuroscience modeling. Furthermore, we uncover some of the mechanisms that help these networks avoid catastrophic forgetting and generalize to unseen tasks, producing hypothesis as to how biological brains achieve these properties.