A Comparative Study and Analysis of Conversational Search Algorithms to Improve their Reproducibility

Conversational Search is a field of Information Retrieval that is steadily gaining popularity in recent years. A conversational retrieval system aims to engage with the users in conversations using natural language. In this work, we studied, implemented and compared a total of eleven state-of-the-art algorithms and strategies for conversational search. Additionally, we developed a conversational retrieval framework focused on modularity, extensibility and reproducibility, that we used to test said algorithms. The top-performant method we tested, Context Query, obtained an nDCG@3 of 0.43, beating more complex methods, like the ones based on coreference resolution or Large Language models (i.e., BERT), by at least 10%. Concerning the reproducibility aspect, we’ve been able to reach comparable results on several methods for which we had a suitable comparing value.