Analyzing Valence Shifts in Scientific Narratives on Psychedelics using Large Language Models and Knowledge Graphs

Once solely categorized as Schedule I drugs, ketamine and psilocybin have resurfaced as promis- ing psychiatric treatments over the last 20 years, leading to a ”psychedelic renaissance” in clinical research. However, there has been no systematic measurement of the development of scientific discourse around these compounds. This thesis therefore presents a novel two-step method based on Artificial Intelligence tools to examine whether individual scientific claims are posi- tive or negative, ranging from 0 (focusing on risks or ”abusive”) to 1 (focusing on benefits or ”therapeutic”), covering 269 peer-reviewed studies on ketamine and psilocybin published be- tween 1995 and 2024. We use a hybrid natural language processing workflow to automatically extract claims from the Abstract, Introduction, and Discussion sections of the articles. These claims are then scored in two ways: (1) GPT-4 gives them ratings based on specific prompts, and (2) Facebook/BART-large-MNLI evaluates them to determine if they are ”therapeutic,” ”neutral,” or ”abusive.” The whole dataset is put into a Neo4j knowledge graph, which helps analyze changes over time and run organized searches, after combining the sentence-level scores into overall scores for each paper. Our results indicate that different models agree well with each other (Pearson’s r ≈ 0.76, with Cronbach’s α ≈ 0.86), indicating strong linear correlation, rank concordance, and internal consistency across models, that there has been an early trend towards using positive language in important journals, and that there has been a significant increase in the use of therapeutic language starting around 2010, which aligns with major clinical trials. In addition to charting the framing of psychedelics research over time, this work creates a reusable framework for valence analysis in other biomedical domains.