CatBoost untuk Deteksi Worm dengan Dataset UNSW-NB15 menggunakan MDI-VAE Untuk Memecahkan Masalah Label yang Tidak Seimbang
Abstract
Abstract Worm attack labels are present in the UNSW-NB15 dataset, but the imbalance condition is severe. The purpose of this study is to analyze the effectiveness of the variational autoencoder (VAE) in addressing the imbalance condition in the UNSW-NB15 dataset. This study uses one of the minority labels in the UNSW-NB15 dataset, the worm label. In the next stage, this study performed feature selection using mean decrease in impurity (MDI) with a baseline ExtraTrees classifier (ETC) and compared it with the stateof- the-art method, random forest (RF). We used accuracy, precision, recall, and F1-score to compare the two feature selection methods. We then proposed an ensemble model based on category boosting (CatBoost) and compared its performance with adaptive boosting (AdaBoost) using sensitivity, specificity, and the geometric mean (G-mean). In the final stage, we evaluated the effect of VAE in generating synthetic data to balance the training data. Test results showed that CatBoost using MDI ETC outperformed MDI RF, achieving precision and G-mean scores of 0.80 and 0.86, respectively, compared with 0.61 and 0.68, respectively. CatBoost outperformed AdaBoost in sensitivity and G-mean, achieving 0.97 and 0.99, respectively, compared with 0.67 and 0.82 for AdaBoost. Finally, the application of VAE had no significant effect on CatBoost’s AUC, whereas MDI+VAE increased that score from 0.987 to 0.995. Keywords: : intrusion detection systems, class imbalance, variational autoencoder, UNSW-NB15, categorical boosting
Downloads
References
[1] M. S. Al-Daweri, K. A. Zainol Ariffin, S. Abdullah, and M. F. E. Md. Senan, “An Analysis of the KDD99 and UNSW-NB15 Datasets for the Intrusion Detection System,” Symmetry, vol. 12, no. 10, p. 1666, Oct. 2020, doi: 10.3390/sym12101666.
[2] S. W. Fernandes, A. Gautama Putrada, R. L. Wicaksono, and F. Dea Reskyadita, “Tomek Links and Random Undersampling for Infiltration Attack Detection in Imbalanced CSE-CIC-IDS2018 Dataset,” in 2025 IEEE International Conference on Communication, Networks and Satellite (COMNETSAT), Padang, Indonesia: IEEE, Dec. 2025, pp. 272–277. doi: 10.1109/COMNETSAT68601.2025.11324919.
[3] H. Xie, Y. Shao, Z. Li, Z. Alomari, and A. Makanju, “Optimization of Class Imbalance Techniques in Machine Learning Models for Network Intrusion Detection,” in 2025 9th International Conference on Cryptography, Security and Privacy (CSP), Okinawa, Japan: IEEE, Apr. 2025, pp. 102–106. doi: 10.1109/CSP66295.2025.00025.
[4] M. Li, H. Zhou, Q. Liu, Y. Shao, and G. Wang, “INGB: Informed Nonlinear Granular Ball Oversampling Framework for Noisy Imbalanced Classification,” 2023, arXiv. doi: 10.48550/ARXIV.2307.01224.
[5] B. Xie, X. Xu, and G. Wen, “Network Intrusion Detection Optimization based on Graph Neural Networks and Variational Autoencoders,” in 2024 6th International Conference on Frontier Technologies of Information and Computer (ICFTIC), Qingdao, China: IEEE, Dec. 2024, pp. 127–134. doi: 10.1109/ICFTIC64248.2024.10912964.
[6] S. Tahir, “Variational Autoencoders (VAEs) for Anomaly Detection:,” in Advances in Digital Crime, Forensics, and Cyber Terrorism, N. Z. Jhanjhi, Ed., IGI Global, 2024, pp. 309–326. doi: 10.4018/979-8-3693-8944-7.ch009.
[7] Safana Hyder Abbas, Wedad Abdul Khuder Naser, and Amal Abbas Kadhim, “Subject review: Intrusion Detection System (IDS) and Intrusion Prevention System (IPS),” Glob. J. Eng. Technol. Adv., 10.30574/gjeta.2023.14.2.0031.
[8] N. Hassouneh and S. Al-sharaeh, “Intrusion Detection in IoT Networks Using LSTM Deep Learning Models with the UNSW-NB15 Dataset,” in 2025 International Conference on New Trends in Computing Sciences (ICTCS), Amman, Jordan: IEEE, Apr. 2025, pp. 263–269. doi: 10.1109/ICTCS65341.2025.10989358.
[9] M. Zeeshan et al., “Protocol-Based Deep Intrusion Detection for DoS and DDoS Attacks Using UNSW-NB15 and Bot-IoT Data-Sets,” IEEE Access, vol. 10, pp. 2269–2283, 2022, doi: 10.1109/ACCESS.2021.3137201.
[10] Z. Zoghi and G. Serpen, “Building an intrusion detection system on UNSW ‐ NB15 : Reducing the margin of error to deal with data overlap and imbalance,” Concurr. Comput. Pract. Exp., vol. 36, no. 25, p. e8242, Nov. 2024, doi: 10.1002/cpe.8242.
[11] Y.-D. Lin, Z.-Q. Liu, R.-H. Hwang, V.-L. Nguyen, P.-C. Lin, and Y.-C. Lai, “Machine Learning With Variational AutoEncoder for Imbalanced Datasets in Intrusion Detection,” IEEE Access, vol. 10, pp. 15247–15260, 2022, doi: 10.1109/ACCESS.2022.3149295.
[12] W. Tian, Y. Shen, N. Guo, J. Yuan, and Y. Yang, “VAE-WACGAN: An Improved Data Augmentation Method Based on VAEGAN for Intrusion Detection,” Sensors, vol. 24, no. 18, p. 6035, Sep. 2024, doi: 10.3390/s24186035.
[13] D. G. Edelberg and R. R. Lederman, “Using VAEs to Learn Latent Variables: Observations on Applications in cryo-EM,” 2023, arXiv. doi: 10.48550/ARXIV.2303.07487.
[14] “1244_A_prescriptive_theory_for.”
[15] S. Dodda, “Exploring Variational Autoencoders and Generative Latent Time-Series Models for Synthetic Data Generation and Forecasting,” in 2024 Control Instrumentation System Conference (CISCON), Manipal, India: IEEE, Aug. 2024, pp. 1–6. doi: 10.1109/CISCON62171.2024.10696588.
[16] G. Alfian et al., “Predicting Breast Cancer from Risk Factors Using SVM and Extra-Trees-Based Feature Selection Method,” Computers, vol. 11, no. 9, p. 136, Sep. 2022, doi: 10.3390/computers11090136.
[17] S. Geeitha, K. Ravishankar, J. Cho, and S. V. Easwaramoorthy, “Integrating cat boost algorithm with triangulating feature importance to predict survival outcome in recurrent cervical cancer,” Sci. Rep., vol. 14, no. 1, p. 19828, Aug. 2024, doi: 10.1038/s41598-024-67562-0.
[18] M. A. Bouke, A. Abdullah, J. Frnda, K. Cengiz, and B. Salah, “BukaGini: A Stability-Aware Gini Index Feature Selection Algorithm for Robust Model Performance,” IEEE Access, vol. 11, pp. 59386– 59396, 2023, doi: 10.1109/ACCESS.2023.3284975.
[19] R. Hornung, “Diversity Forests: Using Split Sampling to Enable Innovative Complex Split Procedures in Random Forests,” SN Comput. Sci., vol. 3, no. 1, p. 1, Jan. 2022, doi: 10.1007/s42979-021-00920-1.
[20] N. Fauzi, F. A. Yulianto, and H. H. Nuha, “The Effectiveness of Anomaly-Based Intrusion Detection Systems in Handling Zero-Day Attacks Using AdaBoost, J48, and Random Forest Methods,” in 2023 IEEE Asia Pacific Conference on Wireless and Mobile (APWiMob), Bali, Indonesia: IEEE, Oct. 2023, pp. 57– 62. doi: 10.1109/APWiMob59963.2023.10365642.
[21] K. Filom, A. Miroshnikov, K. Kotsiopoulos, and A. R. Kannan, “On marginal feature attributions of tree-based models,” Found. Data Sci., vol. 6, no. 4, pp. 395–467, 2024, doi: 10.3934/fods.2024021.
[22] J. A. Amien, Yoze Rizki, and Mukhlis Ali Rahman Nasution, “Implementasi Adasyn Untuk Imbalance Data Pada Dataset UNSW-NB15 Adasyn Implementation For Data Imbalance on UNSW-NB15 Dataset,” J. CoSciTech Comput. Sci. Inf. Technol., vol. 3, no. 3, pp. 242–248, Dec. 2022, doi: 10.37859/coscitech.v3i3.4339.
[23] Y. Wang, Y. Wei, and H. Wang, “A class imbalanced wafer defect classification framework based on variational autoencoder generative adversarial network,” Meas. Sci. Technol., vol. 34, no. 2, p. 024008, Feb. 2023, doi: 10.1088/1361-6501/ac9ed3.
