Pengembangan Sistem Pelabelan Otomatis Untuk Aspect-Based Sentiment Analysis Pada Ulasan Video Game Menggunakan Large Language Models
Abstract
Industri video game berkembang pesat dengan jutaan ulasan pengguna yang dihasilkan setiap tahun, namun volume dan kompleksitas opini multi-aspek dalam ulasan tersebut menyulitkan pengembang untuk memperoleh ringkasan yang terstruktur. Penelitian ini bertujuan untuk merancang dan mengembangkan sebuah sistem prototipe fungsional berupa dashboard untuk mengotomatiskan proses Aspect-Based Sentiment Analysis (ABSA) pada ulasan video game. Metodologi penelitian yang digunakan adalah pendekatan hibrida yang mengintegrasikan CRISP-DM untuk fase riset data dan Model Waterfall untuk fase pengembangan sistem. Sebuah dataset ground truth berkualitas tinggi yang terdiri dari 896 ulasan Steam berhasil dibangun melalui proses anotasi manual yang ketat dan divalidasi dengan reliabilitas antar-anotator yang sangat tinggi (nilai Fleiss' Kappa rata-rata > 0.85). Eksperimen evaluasi komprehensif dilakukan untuk menemukan kombinasi strategi prompt engineering dan Large Language Model (LLM) yang paling optimal. Hasil penelitian menunjukkan bahwa strategi prompt 4-Shot + Guideline merupakan yang paling efektif. Pada evaluasi perbandingan model, Llama 4 Maverick menunjukkan kinerja tertinggi untuk tugas Aspect Category Detection (ACD) dengan Macro F1-Score 0.9055 dan tugas Aspect Category Sentiment Classification (ACSC) dengan Macro F1-Score 0.8373. Temuan ini diimplementasikan dalam sebuah prototipe dashboard fungsional, yang menunjukkan kelayakan kombinasi model LLM state-of-the-art dengan strategi prompt yang tepat sebagai pendekatan yang menjanjikan untuk menghasilkan analisis opini pemain yang terstruktur secara otomatis.
Kata kunci— analisis sentimen berbasis aspek, large language model, prompt engineering, ulasan video game, pengembangan sistem.
References
“Meet the Moment: How Gamers Are Changing the Game,” 2024.
T. Kraemer, W. H. Weiger, and S. Heidenreich, “Do all stars shine the same? Investigating the nonlinear effects of user and critic reviews on video game sales,” J Bus Res, vol. 188, Feb. 2025, doi: 10.1016/j.jbusres.2024.115034.
M. Philp and M. V. Nepomuceno, “How reviews influence product usage post-purchase: An examination of video game playtime,” J Bus Res, vol. 172, Feb. 2024, doi: 10.1016/j.jbusres.2023.114456.
A. Kosmopoulos, A. Liapis, G. Giannakopoulos, and N. Pittaras, “Summarizing Game Reviews: First Contact,” 2020. [Online]. Available: www.metacritic.com
Y. C. Hua, P. Denny, K. Taskova, and J. Wicker, “A Systematic Review of Aspect-based Sentiment Analysis: Domains, Methods, and Trends,” Nov. 2023, doi: 10.1007/s10462-024-10906-z.
I. A. Kandhro, F. Ali, M. Uddin, A. Kehar, and S. Manickam, “Exploring aspect-based sentiment analysis: an in-depth review of current methods and prospects for advancement,” Jul. 01, 2024, Springer Science and Business Media Deutschland GmbH. doi: 10.1007/s10115-024-02104-8.
P. F. Simmering and P. Huoviala, “Large language models for aspect-based sentiment analysis,” no. 2022, pp. 1–12, 2023, [Online]. Available: http://arxiv.org/abs/2310.18025
K. Scaria, H. Gupta, S. Goyal, S. A. Sawant, S. Mishra, and C. Baral, “InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis,” Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2024, vol. 2, pp. 720–736, 2024, doi: 10.18653/v1/2024.naacl-short.63.
H. T. Ismet, T. Mustaqim, and D. Purwitasari, “Aspect Based Sentiment Analysis of Product Review Using Memory Network,” Scientific Journal of Informatics, vol. 9, no. 1, pp. 73–83, May 2022, doi: 10.15294/sji.v9i1.34094.
W. Zhang, X. Li, Y. Deng, L. Bing, and W. Lam, “A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges,” IEEE Trans Knowl Data Eng, vol. 35, no. 11, pp. 11019–11038, 2023, doi: 10.1109/TKDE.2022.3230975.
Y. Li et al., “Better Queries for Aspect-Category Sentiment Classification,” 19th Chinese National Conference on Computational Linguistic, CCL 2020, no. c, pp. 1079–1088, 2020.
D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural Language Processing : State of The Art , Current Trends and Challenges,” 2023.
H. Naveed et al., “A Comprehensive Overview of Large Language Models,” 2023, [Online]. Available: http://arxiv.org/abs/2307.06435
F. Adıgüzel, “The Effect of YouTube Reviews on Video Game Sales,” Journal of Business Research - Turk, vol. 13, no. 3, pp. 2096–2109, Sep. 2021, doi: 10.20491/isarder.2021.1249.
M. Viggiato, D. Lin, A. Hindle, and C. P. Bezemer, “What Causes Wrong Sentiment Classifications of Game Reviews,” IEEE Trans Games, 2021, doi: 10.1109/TG.2021.3072545.
J. Al Mursyidy Fadhlurrahman, N. A. Herawati, H. R. Widya Aulya, I. Puspasari, and N. P. Utama, “Sentiment Analysis of Game Reviews on STEAM using BERT, BiLSTM, and CRF,” in Proceedings of the International Conference on Electrical Engineering and Informatics, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/ICEEI59426.2023.10346219.
N. Khanh and H. Tran, “BACHELOR THESIS Development of a Machine Learning System for Aspect-Based Sentiment Analysis and Text Summarization of Video Game Reviews on Steam.” [Online]. Available: https://store.steampowered.com/
M. D. Purbolaksono, U. K. Dewi, R. L. Wicaksono, and A. P. Wibowo, “Sentiment Analysis of Game Review using Random Forest,” vol. 10, no. 2, pp. 161–169, 2024, doi: 10.21108/ijoict.v10i2.1007.
R. Wirth and J. Hipp, “CRISP-DM: Towards a Standard Process Model for Data Mining,” 2000.
S. Mudassar, “Waterfall Model Used in Software Development Reference: Software Requirements Engineering Waterfall Model,” ResearchGate, no. June, pp. 2–4, 2023, doi: 10.13140/RG.2.2.29580.69764.
S. Heidenreich, F. Handrich, and T. Kraemer, “Flawless victory! Investigating search and experience qualities as antecedent predictors of video game success,” Electronic Markets, vol. 33, no. 1, Dec. 2023, doi: 10.1007/s12525-023-00647-2.
E. Guardiola, “Gameplay definition: A game design perspective,” 20th International Conference on Intelligent Games and Simulation, GAME-ON 2019, no. October, pp. 5–10, 2019.
M. S. Balalaa, “Video Game Graphics: A Comprehensive Analysis of Styles and Techniques,” vol. 13, no. 3, 2023.
E. Aarseth, “A narrative theory of games,” Foundations of Digital Games 2012, FDG 2012 - Conference Program, no. October, pp. 129–133, 2012, doi: 10.1145/2282338.2282365.
A. Wrześniewska and M. Skublewska-Paszkowska, “Video game performance analysis on selected operating systems Analiza wydajności gier komputerowych na wybranych systemach operacyjnych,” vol. 29, no. June, pp. 317–324, 2023.
L. Caroux, “Presence in video games: A systematic review and meta-analysis of the effects of game design choices,” Appl Ergon, vol. 107, no. February, 2023, doi: 10.1016/j.apergo.2022.103936.
J. Opitz, “A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice,” Trans Assoc Comput Linguist, vol. 12, no. 2018, pp. 820–836, 2024, doi: 10.1162/tacl_a_00675.
