Exploiting Utterance Permutation in Conversational Search to Improve System Evaluation

In the field of Conversational Information Seeking it is necessary to evaluate and analyze the behavior of conversational systems. In this thesis, I exploit the permutation of the utterances to improve the evaluation of the system. In practice I simulated a conversational search of a real user to create the permutations, in particular I chose each successive utterance in a conversation by reading and considering the response of the conversational system. Finally, I carry out an analysis of the results to explain whether the permutations created have improved the evaluation of the corresponding system.