Comparative Evaluation of Machine Learning Models for Credit Card Approval Prediction Using Class Imbalance Handling Methods (Jurnal JTIK (Jurnal Teknologi Informasi dan Komunikasi))

Summary

• The study comparatively evaluates machine learning models using different class imbalance handling strategies on a dataset of 25,128 credit card applications.p.1
• The original dataset is highly imbalanced, comprising 25,007 approved applications (99.52%) and only 121 rejected applications (0.48%).p.4
• Under PyCaret's built-in SMOTE scenario, Random Forest achieved an accuracy of 99.68% on the holdout dataset, but its recall for the rejected class was only 36.11%.p.5
• Evaluating performance with balanced data yielded a Cohen's Kappa of 0.9726 and a Matthews Correlation Coefficient of 0.9730 for the Gradient Boosting model.p.6
• Feature importance analysis revealed that Years_of_Working contributed 83.06% and Total_Bad_Debt contributed 14.35% to the final prediction model.p.8

Key insights

1. High overall predictive accuracy can conceal poor minority-class detection in highly imbalanced credit card approval datasets.p.5
2. PyCaret's built-in SMOTE imbalance handling left the model strongly biased toward approved applications, correctly classifying only 13 of 36 rejected applications in holdout testing.p.5
3. Complementary metrics such as Cohen's Kappa and Matthews Correlation Coefficient provide a more reliable assessment than accuracy alone under severe class imbalance.p.7
4. Employment duration (Years_of_Working) dominated model decision-making, accounting for 83.06% of total feature importance.p.8
5. Debt-related risk (Total_Bad_Debt) was the second most significant predictor, representing 14.35% of feature importance.p.8
6. Dominant predictors like long employment history can outweigh elevated bad debt indicators in borderline misclassification cases.p.8