ChatGPT and Simple Text-2024: evaluating ChatGPT3.5 on terminology tasks

This dissertation investigates the capabilities of ChatGPT3.5 in performing terminology-related tasks within the framework of the CLEF Automatic Simplification of Scientific Texts (SimpleText) 2024 project. Specifically, this work aims to assess ChatGPT 3.5’s proficiency in identifying terms, evaluating their difficulty levels, formulating intensional definitions, and simplifying or explaining existing definitions. To achieve this, the dissertation employs a comprehensive evaluation methodology, incorporating both qualitative and quantitative analysis techniques, and a diverse dataset of scientific texts is utilized to assess ChatGPT 3.5’s performance across various terminology-related tasks. The evaluation results are analysed to identify strengths, weaknesses, and areas for improvement in ChatGPT 3.5’s capabilities. Specifically, the findings of this dissertation aim to contribute to a deeper understanding of ChatGPT 3.5’s potential (and limitations) as a tool for terminology-related tasks within the context of scientific text simplification, informing future research and development efforts aimed at enhancing the capabilities of large language models for applications in the field of terminology and language technology.