Comparative Evaluation of Post-Hoc Feature Attribution Methods on Tabular Financial Data: Faithfulness, Stability, and Computational Efficiency
Keywords:
explainable artificial intelligence, feature attribution, credit scoring, faithfulness evaluationAbstract
The deployment of machine learning in credit scoring and fraud detection has intensified regulatory and societal demand for transparent decision-making. Post-hoc feature attribution methods such as SHAP, LIME, Integrated Gradients, and Anchors promise to explain individual predictions, yet their comparative reliability on financial tabular data remains insufficiently characterized. This study conducts a controlled empirical evaluation of four prominent attribution methods across four public financial datasets spanning credit scoring and transaction fraud detection. Three classifiers---XGBoost, Random Forest, and Multilayer Perceptron---serve as the underlying predictive functions. Explanation quality is quantified along three axes: faithfulness measured by Prediction Gap on Important features and infidelity, stability measured by max-sensitivity, and computational efficiency measured by wall-clock time per explanation. Results indicate that TreeSHAP achieves the highest faithfulness and lowest sensitivity on tree-based classifiers, while Integrated Gradients attains competitive faithfulness on neural networks. LIME exhibits the largest variance across repeated runs, raising concerns for regulatory settings that require reproducible explanations. Anchors produce the sparsest explanations at the cost of reduced faithfulness. No single method dominates all evaluation criteria simultaneously, corroborating recent theoretical predictions of an inherent trade-off among explanation desiderata. These findings provide practitioners and regulators with empirically grounded guidance for selecting attribution methods in financial applications.References
1. Bücker, M., Szepannek, G., Gosiewska, A., and Biecek, P., "Transparency, auditability, and explainability of machine learning models in credit scoring," Journal of the Operational Research Society, vol. 73, no. 1, pp. 70--90, 2022.
2. Lundberg, S. M., and Lee, S.-I., "A unified approach to interpreting model predictions," in Advances in Neural Information Processing Systems 30 (NeurIPS 2017), pp. 4765--4774, 2017.
3. Ribeiro, M. T., Singh, S., and Guestrin, C., "'Why should I trust you?': Explaining the predictions of any classifier," in *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016)*, pp. 1135--1144, 2016.
4. Krishna, S., Han, T., Gu, A., Pombra, J., Jabbari, S., Wu, S., and Lakkaraju, H., "The disagreement problem in explainable machine learning: A practitioner's perspective," Transactions on Machine Learning Research, 2024.
5. Alvarez-Melis, D., and Jaakkola, T. S., "Towards robust interpretability with self-explaining neural networks," in Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pp. 7775--7784, 2018.
6. Sundararajan, M., Taly, A., and Yan, Q., "Axiomatic attribution for deep networks," in Proceedings of the 34th International Conference on Machine Learning (ICML 2017), PMLR 70, pp. 3319--3328, 2017.
7. Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B., "Sanity checks for saliency maps," in Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pp. 9505--9515, 2018.
8. Yeh, C.-K., Hsieh, C.-Y., Suggala, A., Inouye, D. I., and Ravikumar, P. K., "On the (in)fidelity and sensitivity of explanations," in Advances in Neural Information Processing Systems 32 (NeurIPS 2019), pp. 10965--10976, 2019.
9. Hooker, S., Erhan, D., Kindermans, P.-J., and Kim, B., "A benchmark for interpretability methods in deep neural networks," in Advances in Neural Information Processing Systems 32 (NeurIPS 2019), pp. 9734--9745, 2019.
10. Agarwal, C., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Puri, I., Zitnik, M., and Lakkaraju, H., "OpenXAI: Towards a transparent evaluation of model explanations," in Advances in Neural Information Processing Systems 35 (NeurIPS 2022), Datasets and Benchmarks Track, 2022.
11. Han, T., Srinivas, S., and Lakkaraju, H., "Which explanation should I choose? A function approximation perspective to characterizing post hoc explanations," in Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 2022.
12. Gramegna, A., and Giudici, P., "SHAP and LIME: An evaluation of discriminative power in credit risk," Frontiers in Artificial Intelligence, vol. 4, Article 752558, 2021.
13. Bracke, P., Datta, A., Jung, C., and Sen, S., "Machine learning explainability in finance: An application to default risk analysis," Bank of England Staff Working Paper No. 816, 2019.
14. Misheva, B. H., Osterrieder, J., Hirsa, A., Kulkarni, O., and Lin, S., "Explainable AI in credit risk management," arXiv:2103.00949, 2021.
15. Rong, Y., Leemann, T., Borisov, V., Kasneci, G., and Kasneci, E., "A consistent and efficient evaluation strategy for attribution methods," in Proceedings of the 39th International Conference on Machine Learning (ICML 2022), PMLR 162, 2022.
16. Hedström, A., Weber, L., Krakowczyk, D., Bareeva, D., Motzkus, F., Samek, W., Lapuschkin, S., and Höhne, M. M.-C., "Quantus: An explainable AI toolkit for responsible evaluation of neural network explanations and beyond," Journal of Machine Learning Research, vol. 24, no. 34, pp. 1--11, 2023.
17. Li, X., Du, M., Chen, J., Chai, Y., Xiong, H., and Lakkaraju, H., "M4: A unified XAI benchmark for faithfulness evaluation of feature attribution methods across metrics, modalities and models," in Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Datasets and Benchmarks Track, 2023.
18. Doshi-Velez, F., and Kim, B., "Towards a rigorous science of interpretable machine learning," arXiv:1702.08608, 2017.
