A Comparative Evaluation of Class Imbalance Handling Techniques for Credit Card Fraud Detection
Keywords:
class imbalance, fraud detection, resampling techniques, ensemble learningAbstract
Credit card fraud detection presents a challenging classification task due to extreme class imbalance, where fraudulent transactions constitute less than 1% of all observations. Selecting an appropriate imbalance handling technique is critical, yet the comparative performance of these techniques under varying imbalance severities remains insufficiently understood. This study conducts a systematic empirical evaluation of nine class imbalance handling techniques across two publicly available fraud detection datasets exhibiting different imbalance ratios (578:1 and 90:1). The techniques evaluated span data-level resampling (SMOTE, ADASYN, Borderline-SMOTE, SMOTE combined with Edited Nearest Neighbors, and random undersampling), algorithm-level cost-sensitive learning (class weighting), and ensemble-based approaches (EasyEnsemble, RUSBoost, and Balanced Random Forest). Each technique is paired with four base classifiers---Logistic Regression, Random Forest, XGBoost, and LightGBM---and assessed using five evaluation metrics: AUC-ROC, PR-AUC, F1-score, Matthews Correlation Coefficient, and recall. Results indicate that ensemble-based methods, particularly EasyEnsemble, achieve the most consistent improvements across both datasets. Hybrid resampling via SMOTE with Edited Nearest Neighbors produces comparable gains among data-level methods. A notable finding is that standard SMOTE, while improving AUC-ROC and F1-score, can reduce PR-AUC relative to the untreated baseline under severe imbalance. Cost-sensitive class weighting emerges as a computationally efficient alternative that preserves strong PR-AUC performance. These findings provide practical guidance for practitioners selecting imbalance handling strategies in fraud detection applications.References
1. H. He and E. A. Garcia, "Learning from imbalanced data," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, pp. 1263–1284, 2009.
2. A. Dal Pozzolo, G. Boracchi, O. Caelen, C. Alippi, and G. Bontempi, "Credit card fraud detection: A realistic modeling and a novel learning strategy," IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 8, pp. 3784–3797, 2018.
3. J. Davis and M. Goadrich, "The relationship between Precision-Recall and ROC curves," in Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240, 2006.
4. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic minority over-sampling technique," Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002.
5. H. He, Y. Bai, E. A. Garcia, and S. Li, "ADASYN: Adaptive synthetic sampling approach for imbalanced learning," in Proceedings of the IEEE International Joint Conference on Neural Networks, pp. 1322–1328, 2008.
6. H. Han, W.-Y. Wang, and B.-H. Mao, "Borderline-SMOTE: A new over-sampling method in imbalanced data sets learning," in Advances in Intelligent Computing, LNCS vol. 3644, pp. 878–887, Springer, 2005.
7. G. E. A. P. A. Batista, R. C. Prati, and M. C. Monard, "A study of the behavior of several methods for balancing machine learning training data," ACM SIGKDD Explorations Newsletter, vol. 6, no. 1, pp. 20–29, 2004.
8. C. Elkan, "The foundations of cost-sensitive learning," in *Proceedings of the 17th International Joint Conference on Artificial Intelligence*, pp. 973–978, 2001.
9. P. Domingos, "MetaCost: A general method for making classifiers cost-sensitive," in *Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, pp. 155–164, 1999.
10. S. Jesus, J. Pombal, D. Alves, A. Cruz, P. Saleiro, R. Ribeiro, J. Gama, and P. Bizarro, "Turning the tables: Biased, imbalanced, dynamic tabular datasets for ML evaluation," in Advances in Neural Information Processing Systems 35, 2022.
11. T. Chen and C. Guestrin, "XGBoost: A scalable tree boosting system," in *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, pp. 785–794, 2016.
12. G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, "LightGBM: A highly efficient gradient boosting decision tree," in Advances in Neural Information Processing Systems 30, pp. 3146–3154, 2017.
13. X.-Y. Liu, J. Wu, and Z.-H. Zhou, "Exploratory undersampling for class-imbalance learning," IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 39, no. 2, pp. 539–550, 2009.
14. N. V. Chawla, A. Lazarevic, L. O. Hall, and K. W. Bowyer, "SMOTEBoost: Improving prediction of the minority class in boosting," in *Proceedings of the 7th European Conference on Principles and Practice of Knowledge Discovery in Databases*, pp. 107–119, 2003.
15. C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano, "RUSBoost: A hybrid approach to alleviating class imbalance," IEEE Transactions on Systems, Man, and Cybernetics---Part A, vol. 40, no. 1, pp. 185–197, 2010.
16. Z.-H. Zhou and X.-Y. Liu, "Training cost-sensitive neural networks with methods addressing the class imbalance problem," IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 1, pp. 63–77, 2006.
17. K. Cao, C. Wei, A. Gaidon, N. Aréchiga, and T. Ma, "Learning imbalanced datasets with label-distribution-aware margin loss," in Advances in Neural Information Processing Systems 32, pp. 1565–1576, 2019.
18. Y. Ma, Y. Tian, N. Moniz, and N. V. Chawla, "Class-imbalanced learning on graphs: A survey," ACM Computing Surveys, vol. 57, no. 8, Article 207, 2025.
