Topic modeling in chat transcripts from an e-commerce website

Abstract

This thesis investigates the application of topic modeling, specifically through the use of Latent Dirichlet Allocation (LDA), to analyze customer service chat transcripts from an e-commerce platform. The aim is to identify latent topics within the chat data that can offer insights into customer behaviors, preferences, and frequently encountered issues. The methodology includes a comprehensive data preprocessing phase, including the anonymization of personal information, tokenization of text, and removal of stopwords and irrelevant characters to prepare for LDA model training. The LDA model is calibrated across several iterations to identify the most coherent topic distribution, with the optimal number of topics determined by evaluating model coherence scores. Two different models are trained to provide a different number of identified topics. The final selection is based on the balance of detail and coherence in the topics. The analysis revealed distinct topics that include various aspects of customer interaction, such as payment issues, shipping information, technical support, and product inquiries. These topics not only reflect the diverse range of customer service inquiries but also highlight specific areas for potential improvement in product offerings, delivery, and website functionality. The findings of this study underscore the utility of topic modeling in discovering valuable insights from textual data, suggesting its applicability in enhancing the strategic decision-making process in the field of e-commerce. This contribution enriches the body of natural language processing applications in business intelligence, proposing an approach to leveraging unstructured data for strategic advantage