A Comparative Study of Naïve Bayes, SVM, Random Forest, and LSTM Performance in Sentiment Analysis on a Movie Review Dataset
Article Sidebar
Main Article Content
Abstract
Sentiment analysis is an important task in natural language processing, aimed at identifying and classifying opinions or emotions in textual data. This study compares the performance of four classification algorithms—Naïve Bayes, Support Vector Machine (SVM), Random Forest, and Long Short-Term Memory (LSTM)—on 25,000 English-language movie reviews with balanced sentiment labels. Text preprocessing includes cleaning, tokenization, and TF-IDF vectorization for traditional models. For LSTM, both randomly initialized embeddings and pre-trained embeddings are tested. Results, evaluated using accuracy, F1-score, and confusion matrix, show that SVM performs best with 89% accuracy, followed by Naïve Bayes and LSTM at 86%, and Random Forest at 82%. LSTM performs poorly with TF-IDF or self-trained embeddings but improves significantly with pre-trained embeddings. These findings indicate that traditional models, especially SVM, remain highly effective for sentiment analysis on moderately sized datasets, while LSTM requires proper text representation to perform competitively.
Article Details
References
[2] B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Found. Trends Inf. Retr., vol. 2, no. 1–2, pp. 1–135, 2008.
[3] H. Basri, M. B. S. Junianto, and I. Kusyadi, “Enhancing Usability Testing Through Sentiment Analysis: A Comparative Study Using SVM, Naive Bayes, Decision Trees and Random Forest,” J. Teknol. Sistem Inform. Apl., vol. 7, no. 4, pp. 1603–1610, Oct. 2024.
[4] N. Febriyanti and A. F. Rozi, “Komparasi Algoritma Naïve Bayes, Support Vector Machine, dan Random Forest untuk Analisis Sentimen Ulasan Pengguna Aplikasi CGV Cinemas Indonesia,” BITS, vol. 7, no. 1, pp. 27–34, 2022.
[5] L. Breiman, “Random Forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001.
[6] K. Pandit, A. Patel, R. Shah, and S. Jain, “Comparative Analysis of Deep Learning Models for Sentiment Analysis on IMDB Reviews,” J. Electr. Syst., vol. 20, no. 2s, 2024
[7] M. Kayed, R. P. Díaz-Redondo, and A. Mabrouk, “Deep Learning-based Sentiment Classification: A Comparative Survey,” arXiv preprint arXiv:2312.17253, Dec. 2023.
[8] G. Nkhata, S. Gauch, U. Anjum, and J. Zhan, “Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis,” arXiv preprint arXiv:2502.20682, Feb. 2025.
[9] Z. Zhao et al., “Sentiment Analysis Based on Deep Learning: A Comparative Study,” Electronics, vol. 9, no. 3, pp. 483–499, Mar. 2020.
[10] R. Garg and A. K. Saha, “An Exhaustive Comparative Study of Machine Learning Algorithms for Natural Language Processing Applications,” Electronics, vol. 12, no. 1, pp. 118–134, Jan. 2023.
[11] L. . Damayanti and K. M. . Lhaksmana, “Sentiment Analysis of the 2024 Indonesia Presidential Election on Twitter”, SinkrOn, vol. 8, no. 2, pp. 938-946, Mar. 2024.
[12] L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using Bert Language Models”, J. Ilm. Tek. Elektro Komput. Dan Inform, vol. 9, no. 3, pp. 746–757, Aug. 2023.
[13] T. C. Herdiyani and A. U. Zailani, "Sentiment analysis terkait pemindahan Ibu Kota Indonesia menggunakan metode Random Forest berdasarkan tweet warga negara Indonesia," Jurnal Teknologi Sistem Informasi, vol. 3, no. 2, pp. 154–165, 2022.
[14] R. Sanusi, F. D. Astuti, and I. Y. Buryadi, "Analisis sentimen pada Twitter terhadap program kartu pra kerja dengan recurrent neural network," JIKO (Jurnal Informatika dan Komputer), vol. 5, no. 2, pp. 89–99, 2021.
[15] T. Sabrila, Y. Azhar, and C. Aditya, "Analisis sentimen tweet tentang UU Cipta Kerja menggunakan algoritma SVM berbasis PSO," JISKA (Jurnal Informatika Sunan Kalijaga), vol. 7, pp. 10–19, 2022, doi: 10.14421/jiska.2022.7.1.10-19.
[16] G. Nkhata, A. K. Bansal, J. H. Oduor, and H. S. Chae, “Performance evaluation of BERT with BiLSTM on IMDB reviews,” arXiv preprint arXiv:2502.20682, Feb. 2025.
[17] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in Proc. 49th Annu. Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA, Jun. 2011, pp. 142–150.