Social Deduction Games are multiplayer games in which players are assigned secret roles and must reason about the intentions, identities, and claims of others. They therefore provide a useful testbed for analyzing the behavior of Large Language Models in environments where information exchange, uncertainty, and persuasive language coexist. This thesis places three Gemma 4 variants of different sizes — 31B, E4B and E2B — as observers of human gameplay in One Night Ultimate Werewolf (ONUW). The models read complete game transcripts, identify the player most likely to be a Werewolf or judge that none is present, and provide a short justification. All three perform above chance, but none is reliably more accurate than the others, despite substantial differences in how reliably they apply the game rules under controlled conditions. A first analysis associates the models' votes with persuasive properties of the discussion. Looking only at the techniques each player uses provides little information about the choices the models make. What is said about a player is more informative: across all three variants, the strongest association is with how often that player is explicitly accused of being a Werewolf. Players who claim the Werewolf role themselves are also more likely to be selected. The justifications are then analyzed both for the relations connecting their statements and for the kind of information they invoke. Here the models differ sharply. 31B makes much greater use of player testimony, comparisons between claims, and the rules of the game, with its mechanical content often taking the form of a conditional reconstruction of the hidden state from reported information. The smaller variants invoke game mechanics far less, appealing instead to the goal of the vote and to the suspicion already expressed by other players. This difference in stated reasoning brings no corresponding accuracy advantage: 31B identifies Werewolves no more accurately than the other variants. Fine-tuning improves state reconstruction when the models are given the information needed to recover the final state, but the improvement does not carry over when that state must instead be inferred from the discussion. E2B and E4B are then re-evaluated on the downstream voting task, where neither improves in accuracy. E4B nevertheless shifts strongly toward judging that no Werewolf is present, and its justifications refer less to the players' behavior during the discussion.
Social Deduction Games are multiplayer games in which players are assigned secret roles and must reason about the intentions, identities, and claims of others. They therefore provide a useful testbed for analyzing the behavior of Large Language Models in environments where information exchange, uncertainty, and persuasive language coexist. This thesis places three Gemma 4 variants of different sizes — 31B, E4B and E2B — as observers of human gameplay in One Night Ultimate Werewolf (ONUW). The models read complete game transcripts, identify the player most likely to be a Werewolf or judge that none is present, and provide a short justification. All three perform above chance, but none is reliably more accurate than the others, despite substantial differences in how reliably they apply the game rules under controlled conditions. A first analysis associates the models' votes with persuasive properties of the discussion. Looking only at the techniques each player uses provides little information about the choices the models make. What is said about a player is more informative: across all three variants, the strongest association is with how often that player is explicitly accused of being a Werewolf. Players who claim the Werewolf role themselves are also more likely to be selected. The justifications are then analyzed both for the relations connecting their statements and for the kind of information they invoke. Here the models differ sharply. 31B makes much greater use of player testimony, comparisons between claims, and the rules of the game, with its mechanical content often taking the form of a conditional reconstruction of the hidden state from reported information. The smaller variants invoke game mechanics far less, appealing instead to the goal of the vote and to the suspicion already expressed by other players. This difference in stated reasoning brings no corresponding accuracy advantage: 31B identifies Werewolves no more accurately than the other variants. Fine-tuning improves state reconstruction when the models are given the information needed to recover the final state, but the improvement does not carry over when that state must instead be inferred from the discussion. E2B and E4B are then re-evaluated on the downstream voting task, where neither improves in accuracy. E4B nevertheless shifts strongly toward judging that no Werewolf is present, and its justifications refer less to the players' behavior during the discussion.
Persuasion and Reasoning in Social Deduction Games: A Study on Large Language Models in One Night Ultimate Werewolf
BRIOTTO, ANNA
2025/2026
Abstract
Social Deduction Games are multiplayer games in which players are assigned secret roles and must reason about the intentions, identities, and claims of others. They therefore provide a useful testbed for analyzing the behavior of Large Language Models in environments where information exchange, uncertainty, and persuasive language coexist. This thesis places three Gemma 4 variants of different sizes — 31B, E4B and E2B — as observers of human gameplay in One Night Ultimate Werewolf (ONUW). The models read complete game transcripts, identify the player most likely to be a Werewolf or judge that none is present, and provide a short justification. All three perform above chance, but none is reliably more accurate than the others, despite substantial differences in how reliably they apply the game rules under controlled conditions. A first analysis associates the models' votes with persuasive properties of the discussion. Looking only at the techniques each player uses provides little information about the choices the models make. What is said about a player is more informative: across all three variants, the strongest association is with how often that player is explicitly accused of being a Werewolf. Players who claim the Werewolf role themselves are also more likely to be selected. The justifications are then analyzed both for the relations connecting their statements and for the kind of information they invoke. Here the models differ sharply. 31B makes much greater use of player testimony, comparisons between claims, and the rules of the game, with its mechanical content often taking the form of a conditional reconstruction of the hidden state from reported information. The smaller variants invoke game mechanics far less, appealing instead to the goal of the vote and to the suspicion already expressed by other players. This difference in stated reasoning brings no corresponding accuracy advantage: 31B identifies Werewolves no more accurately than the other variants. Fine-tuning improves state reconstruction when the models are given the information needed to recover the final state, but the improvement does not carry over when that state must instead be inferred from the discussion. E2B and E4B are then re-evaluated on the downstream voting task, where neither improves in accuracy. E4B nevertheless shifts strongly toward judging that no Werewolf is present, and its justifications refer less to the players' behavior during the discussion.| File | Dimensione | Formato | |
|---|---|---|---|
|
Briotto_Anna.pdf
accesso aperto
Dimensione
3.24 MB
Formato
Adobe PDF
|
3.24 MB | Adobe PDF | Visualizza/Apri |
The text of this website © Università degli studi di Padova. Full Text are published under a non-exclusive license. Metadata are under a CC0 License
https://hdl.handle.net/20.500.12608/116542