Analyzing Graph Out-of-Distribution Generalization through Representational Similarity Measures

This thesis focuses on analyzing graph out-of-distribution (OOD) generalization through representational similarity measures. While the current literature utilizes representational similarity measures to assess model performance, existing analyses mainly consider the in-distribution (ID) setting. In particular, multiple instantiations of the same model are first trained independently, varying only the training seed. This provides several slightly different versions of internal representations and outputs for each model, which are then used to correlate representational and functional similarities. We extend this setting to Graph Neural Networks (GNNs) for OOD node classification tasks by applying it to datasets with carefully designed distribution shifts. Specifically, we test whether the reliability of existing representational similarity measures extends beyond the training data distribution. Moreover, we assess if representations from ID data provide insight into the model’s OOD performance. Our experiments show that, in some cases, the distributions of counts of statistically significant positive correlations between functional and representational similarity measures remain similar across different shifts. However, our results are highly dependent on several factors, such as the nature of the representational similarity measure, the type of OOD shifts, and the datasets used. Hence, further experimentation is necessary to confirm the reliability of our conclusions and the utility of representational similarity measures as a diagnostic tool for graph OOD generalization.