AUDITING OPEN-SOURCE LLMs FOR ACADEMIC AUTHOR SEARCH

This Master’s thesis explores the effectiveness of state-of-the-art Large Language Mod- els (LLMs) in searching for scholars across various fields of physics. The study focuses on auditing the models capabilities in different search-related tasks with a special emphasis on minority representation. These tasks include top-k endorsements, epoch-specific suggestions, field-based recommendations, seniority-related and statistical twins. The thesis discusses the models' strengths, limitations, and potential biases, providing insights for future improvements in academic author search engines. Through rigorous experimentation and data analysis, this work contributes to the broader understanding of LLMs’ application in academic contexts and addresses critical issues of fairness and representation in AI-driven scholarly tools.