Klasifikasi Sentimen pada Dataset Ulasan Film menggunakan Machine Learning dan OpenAI Text Embedding
Abstract
Analisis sentimen pada ulasan film menjadi semakin penting seiring dengan meningkatnya volume data tekstual. Performa model machine learning untuk tugas ini sangat bergantung pada kualitas representasi teks yang digunakan. Penelitian ini bertujuan untuk mengevaluasi efektivitas model embedding teks kontekstual dari OpenAI, Text-embedding-3-large, untuk klasifikasi sentimen pada dataset Movie Reviews. Metodologi penelitian mencakup dua pendekatan klasifikasi: supervised learning menggunakan Support Vector Machine dan Logistic Regression, serta klasifikasi zero-shot. Performa Text-embedding-3-large dibandingkan secara langsung dengan model embedding statis Word2Vec pada dataset yang telah dibersihkan dan dataset asli. Hasil penelitian menunjukkan bahwa Text-embedding-3-large secara signifikan mengungguli Word2Vec, dengan peningkatan F1-score dari 78.01% menjadi 93.20%. Konfigurasi terbaik dicapai oleh kombinasi Support Vector Machine dengan hyperparameter default pada dataset yang tidak dibersihkan, yang mengindikasikan kemampuan model memanfaatkan informasi kontekstual dari tanda baca. Selain itu, pendekatan zero-shot menunjukkan kinerja yang cukup baik dengan F1-score 86.29%, yang membuktikan kapabilitas generalisasi model tanpa memerlukan data latih berlabel.
Kata kunci : klasifikasi sentimen, ulasan film, machine learning, openai, embedding teks, zero-shoT
References
B. Liu, “Sentiment Analysis and Opinion Mining”.
B. Csanády, L. Muzsai, P. Vedres, Z. Nádasdy, dan A. Lukács, “LlamBERT: Large-scale low-cost data annotation in NLP,” 23 Maret 2024, arXiv: arXiv:2403.15938. doi: 10.48550/arXiv.2403.15938.
P. S. Ghatora, S. E. Hosseini, S. Pervez, M. J. Iqbal, dan N. Shaukat, “Sentiment Analysis of Product Reviews Using Machine Learning and Pre-Trained LLM,” BDCC, vol. 8, no. 12, hlm. 199, Des 2024, doi: 10.3390/bdcc8120199.
S. A. Salloum, R. Alfaisal, A. Basiouni, K. Shaalan, dan A. Salloum, “Effectiveness of Logistic Regression for Sentiment Analysis of Tweets About the Metaverse,” dalam Breaking Barriers with Generative Intelligence. Using GI to Improve Human Education and Well-Being, vol. 2162, A. Basiouni dan C. Frasson, Ed., dalam Communications in Computer and Information Science, vol. 2162. , Cham: Springer Nature Switzerland, 2024, hlm. 32–41. doi: 10.1007/978-3-031-65996-6_3.
R. Patil, S. Boit, V. Gudivada, dan J. Nandigam, “A Survey of Text Representation and Embedding Techniques in NLP,” IEEE Access, vol. 11, hlm. 36120–36146, 2023, doi: 10.1109/ACCESS.2023.3266377.
“New embedding models and API updates | OpenAI.” Diakses: 31 Maret 2025. [Daring]. Tersedia pada: https://openai.com/index/new-embedding-models-and-api-updates/
K. Kheiri dan H. Karimi, “SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning,” 23 Juli 2023, arXiv: arXiv:2307.10234. doi: 10.48550/arXiv.2307.10234.
“Vector embeddings - OpenAI API.” Diakses: 21 Maret 2025. [Daring]. Tersedia pada: https://platform.openai.com
Nikhil Sanjay Suryawanshi, “Sentiment analysis with machine learning and deep learning: A survey of techniques and applications,” Int. J. Sci. Res. Arch., vol. 12, no. 2, hlm. 005–015, Jul 2024, doi: 10.30574/ijsra.2024.12.2.1205.
B. Pang dan L. Lee, “Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales,” dalam Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL ’05, Ann Arbor, Michigan: Association for Computational Linguistics, 2005, hlm. 115–124. doi: 10.3115/1219840.1219855.
R. Huang, Q. Chen, J. Tang, dan J. Song, “The Influence of Word Embeddings on the Performance of Sentiment Classification,” IJCIT, vol. 4, no. 1, hlm. 1, Okt 2023, doi: 10.56028/ijcit.1.4.1.2023.
Y. Jin dan A. Zhao, “Bert-based graph unlinked embedding for sentiment analysis,” Complex Intell. Syst., vol. 10, no. 2, hlm. 2627–2638, Apr 2024, doi: 10.1007/s40747-023-01289-9.
A. Deniz, M. Angin, dan P. Angin, “Sentiment and Context-refined Word Embeddings for Sentiment Analysis,” dalam 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Melbourne, Australia: IEEE, Okt 2021, hlm. 927–932. doi: 10.1109/SMC52423.2021.9659189.
M. Zulqarnain, R. Ghazali, M. Aamir, dan Y. M. M. Hassim, “An efficient two-state GRU based on feature attention mechanism for sentiment analysis,” Multimed Tools Appl, vol. 83, no. 1, hlm. 3085–3110, Jan 2024, doi: 10.1007/s11042-022-13339-4.
I. N. Khasanah, “Sentiment Classification Using fastText Embedding and Deep Learning Model,” Procedia Computer Science, vol. 189, hlm. 343–350, 2021, doi: 10.1016/j.procs.2021.05.103.
F. Giglietto, “Evaluating Embedding Models for Clustering Italian Political News: A Comparative Study of Text-Embedding-3-Large and UmBERTo,” 20 Agustus 2024. doi: 10.31219/osf.io/2j9ed.
Z. Huang, Y. Long, K. Peng, dan S. Tong, “An Embedding-Based Semantic Analysis Approach: A Preliminary Study on Redundancy Detection in Psychological Concepts Operationalized by Scales,” J. Intell., vol. 13, no. 1, hlm. 11, Jan 2025, doi: 10.3390/jintelligence13010011.
I. Keraghel, S. Morbieu, dan M. Nadif, “Beyond Words: A Comparative Analysis of LLM Embeddings for Effective Clustering,” dalam Advances in Intelligent Data Analysis XXII, vol. 14641, I. Miliou, N. Piatkowski, dan P. Papapetrou, Ed., dalam Lecture Notes in Computer Science, vol. 14641. , Cham: Springer Nature Switzerland, 2024, hlm. 205–216. doi: 10.1007/978-3-031-58547-0_17.
N. B. Korade, M. B. Salunke, A. A. Bhosle, P. B. Kumbharkar, G. G. Asalkar, dan R. G. Khedkar, “Strengthening Sentence Similarity Identification Through OpenAI Embeddings and Deep Learning,” IJACSA, vol. 15, no. 4, 2024, doi: 10.14569/IJACSA.2024.0150485.
K. Ajroudi, M. I. Khedher, O. Jemai, dan M. A. EI-Yacoubi, “Exploring the Efficacy of Text Embeddings in Early Dementia Diagnosis from Speech,” dalam 2024 16th International Conference on Human System Interaction (HSI), Paris, France: IEEE, Jul 2024, hlm. 1–6. doi: 10.1109/HSI61632.2024.10613581.
S. K. Lho dkk., “Large Language Models and Text Embeddings for Detecting Depression and Suicide in Patient Narratives,” JAMA Netw Open, vol. 8, no. 5, hlm. e2511922, Mei 2025, doi: 10.1001/jamanetworkopen.2025.11922.
D. Venkatesh dan S. Raman, “BITS Pilani at SemEval-2024 Task 1: Using text-embedding-3-large and LaBSE embeddings for Semantic Textual Relatedness,” dalam Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), Mexico City, Mexico: Association for Computational Linguistics, 2024, hlm. 865–868. doi: 10.18653/v1/2024.semeval-1.124.
R. Chandra, J. Sonawane, dan J. Lande, “An Analysis of Vaccine-Related Sentiments on Twitter (X) from Development to Deployment of COVID-19 Vaccines,” BDCC, vol. 8, no. 12, hlm. 186, Des 2024, doi: 10.3390/bdcc8120186.
A. L. Jiménez-Preciado, J. Álvarez-García, S. Cruz-Aké, dan F. Venegas-Martínez, “The Power of Words from the 2024 United States Presidential Debates: A Natural Language Processing Approach,” Information, vol. 16, no. 1, hlm. 2, Des 2024, doi: 10.3390/info16010002.
M. Rodríguez-Ibánez, A. Casánez-Ventura, F. Castejón-Mateos, dan P.-M. Cuenca-Jiménez, “A review on sentiment analysis from social media platforms,” Expert Systems with Applications, vol. 223, hlm. 119862, Agu 2023, doi: 10.1016/j.eswa.2023.119862.
P. Monika, C. Kulkarni, N. Harish Kumar, S. Shruthi, dan V. Vani, “Machine learning approaches for sentiment analysis: A survey,” ijhs, hlm. 1286–1300, Apr 2022, doi: 10.53730/ijhs.v6nS4.6119.
I. Nawawi, K. F. Ilmawan, M. R. Maarif, dan M. Syafrudin, “Exploring Tourist Experience through Online Reviews Using Aspect-Based Sentiment Analysis with Zero-Shot Learning for Hospitality Service Enhancement,” Information, vol. 15, no. 8, hlm. 499, Agu 2024, doi: 10.3390/info15080499.
“Zero-shot classification with embeddings | OpenAI Cookbook.” Diakses: 30 Maret 2025. [Daring]. Tersedia pada: https://cookbook.openai.com/examples/zero-shot_classification_with_embeddings
C. Liu, Y. Sheng, Z. Wei, dan Y.-Q. Yang, “Research of Text Classification Based on Improved TF-IDF Algorithm,” dalam 2018 IEEE International Conference of Intelligent Robotic and Control Engineering (IRCE), Lanzhou: IEEE, Agu 2018, hlm. 218–222. doi: 10.1109/IRCE.2018.8492945.
E. Rudkowsky, M. Haselmayer, M. Wastian, M. Jenny, Š. Emrich, dan M. Sedlmair, “More than Bags of Words: Sentiment Analysis with Word Embeddings,” Communication Methods and Measures, vol. 12, no. 2–3, hlm. 140–157, Apr 2018, doi: 10.1080/19312458.2018.1455817.
M. T. Pilehvar dan J. Camacho-Collados, “Embeddings in Natural Language Processing”.
T. Mikolov, K. Chen, G. Corrado, dan J. Dean, “Efficient Estimation of Word Representations in Vector Space,” 7 September 2013, arXiv: arXiv:1301.3781. doi: 10.48550/arXiv.1301.3781.
J. Pennington, R. Socher, dan C. Manning, “Glove: Global Vectors for Word Representation,” dalam Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar: Association for Computational Linguistics, 2014, hlm. 1532–1543. doi: 10.3115/v1/D14-1162.
A. Vaswani dkk., “Attention Is All You Need,” 2 Agustus 2023, arXiv: arXiv:1706.03762. doi: 10.48550/arXiv.1706.03762.
J. Devlin, M.-W. Chang, K. Lee, dan K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”.
M. Peters dkk., “Deep Contextualized Word Representations,” dalam Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana: Association for Computational Linguistics, 2018, hlm. 2227–2237. doi: 10.18653/v1/N18-1202.
O. Galal, A. H. Abdel-Gawad, dan M. Farouk, “Rethinking of BERT sentence embedding for text classification,” Neural Comput & Applic, vol. 36, no. 32, hlm. 20245–20258, Nov 2024, doi: 10.1007/s00521-024-10212-3.
A. Radford, K. Narasimhan, T. Salimans, dan I. Sutskever, “Improving Language Understanding by Generative Pre-Training”.
T. B. Brown dkk., “Language Models are Few-Shot Learners”.
A. Neelakantan dkk., “Text and Code Embeddings by Contrastive Pre-Training,” 24 Januari 2022, arXiv: arXiv:2201.10005. doi: 10.48550/arXiv.2201.10005.
Q. Li dkk., “A Survey on Text Classification: From Traditional to Deep Learning,” ACM Trans. Intell. Syst. Technol., vol. 13, no. 2, hlm. 1–41, Apr 2022, doi: 10.1145/3495162.
H.-T. Duong dan T.-A. Nguyen-Thi, “A review: preprocessing techniques and data augmentation for sentiment analysis,” Comput Soc Netw, vol. 8, no. 1, hlm. 1, Des 2021, doi: 10.1186/s40649-020-00080-x.
P. R. Amalia dan E. Winarko, “Aspect-Based Sentiment Analysis on Indonesian Restaurant Review Using a Combination of Convolutional Neural Network and Contextualized Word Embedding,” Indonesian J. Comput. Cybern. Syst., vol. 15, no. 3, hlm. 285, Jul 2021, doi: 10.22146/ijccs.67306.
C. Kuo, The handbook of NLP with Gensim: leverage topic modeling to uncover hidden patterns, themes, and valuable insights within textual data, 1st edition. Birmingham, UK: Packt Publishing Ltd., 2023.
O. Ozyurt dan A. Ayaz, “Twenty-five years of education and information technologies: Insights from a topic modeling based bibliometric analysis,” Educ Inf Technol, vol. 27, no. 8, hlm. 11025–11054, Sep 2022, doi: 10.1007/s10639-022-11071-y.
E. Hokijuliandy, H. Napitupulu, dan Firdaniza, “Application of SVM and Chi-Square Feature Selection for Sentiment Analysis of Indonesia’s National Health Insurance Mobile Application,” Mathematics, vol. 11, no. 17, hlm. 3765, Sep 2023, doi: 10.3390/math11173765.
“What is the default threshold in Sklearn logistic regression? - GeeksforGeeks.” Diakses: 16 Juni 2025. [Daring]. Tersedia pada: https://www.geeksforgeeks.org/data-science/what-is-the-default-threshold-in-sklearn-logistic-regression/
