Design and Development of an Extensible and Configurable Framework for Conversational Search Experiments

The Conversational Search (CS) paradigm allows for an intuitive interaction between the user and the system through natural language sentences and it is increasingly being adopted in various scenarios. However, its widespread experimentation has led to the birth of a multitude of CS systems with custom implementations and variants of Information Retrieval (IR) models. This exacerbates the reproducibility crisis already observed in several research areas, including IR. To address this issue, we propose DECAF: a modular and extensible Conversational Search framework designed for fast prototyping and development of conversational agents. Our framework integrates all the components that characterize a modern CS system and allows for the seamless integration of Machine Learning (ML) and Large Language Models (LLMs)-based techniques. Furthermore, thanks to its uniform interface, DECAF allows for experiments characterized by a high degree of reproducibility. DECAF contains several state-of-the-art components including query rewriting, search functions under Bag-of-Words (BoW) and dense paradigms, and re-ranking functions. Our framework is tested on two well-known conversational collections: TREC CAsT 2019 and 2020 and the results can be used by future practitioners as baselines. Our contributions include the identification of a series of state-of-the-art components for the CS task and the definition of a modular framework for its implementation.