Building Secure Conversational Agents: Architectural Choices, Performance Evaluation, and Threat Modeling in RAG

This thesis presents the design, implementation, and cybersecurity evaluation of a secure, onpremise Retrieval-Augmented Generation (RAG) conversational agent tailored for a museum environment. To ensure data sovereignty and operate within a strict 36GB VRAM hardware constraint, the open-weight Qwen3.5-35B-A3B model was selected. The system optimizes latency by decoupling offline document ingestion from real-time generation, employing a hybrid search pipeline that combines Maximal Marginal Relevance (MMR) with BM25, refined by a cross-encoder reranker. This architecture achieves a 72.23% retrieval success rate while maintaining a strict operational latency of 1.4 seconds per query. Crucially, deploying RAG architectures shifts traditional security defenses to a novel ”semantic perimeter”. This study conducts a rigorous threat modeling assessment, evaluating inference- and data-phase attack vectors such as micro-scale data poisoning (PoisonedRAG) , automated jailbreaking (GPTFUZZER) , and Denial of Service exploits. To secure the infrastructure, the research proposes multi-layered countermeasures, including data provenance, retrieval-native access controls, and a Dual LLM pattern