Predicting Customer Churn in Telecommunications: A Controlled Comparison of Machine Learning Models Using Mutual Information Feature Selection
DOI:
https://doi.org/10.64261/hfpwc474Keywords:
customer churn, telecommunications, machine learning, Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, Mutual Information, feature selection, predictive modelingAbstract
Customer churn significantly impacts telecommunications revenue and acquisition costs, but machine learning can proactively identify at-risk customers using historical data. This study compares four supervised algorithms for churn prediction: Logistic Regression, Decision Tree, Random Forest, and Gradient Boosting.
Using a dataset of 7,032 customer records, the study applied a stratified 80:20 train-test split. Preprocessing included median imputation and standardization for numerical features, and most-frequent imputation with one-hot encoding for categorical data. Mutual Information feature selection, using a fixed random state of 42, retained the ten highest-ranked features. All models shared this identical pipeline to ensure a controlled comparison.
Performance was evaluated on an independent holdout test set using accuracy, precision, recall, F1-score, ROC AUC, and confusion matrices, alongside five-fold stratified cross-validation on the training data. On the holdout set, Logistic Regression achieved the highest accuracy (78.82%), while Gradient Boosting recorded 78.61% accuracy and the highest ROC AUC (83.60%), slightly outperforming Logistic Regression (83.39%).
During cross-validation, Gradient Boosting maintained a superior mean ROC AUC of 84.48% ± 0.65%, compared to 84.29% ± 0.52% for Logistic Regression. However, Gradient Boosting identified only 51.60% of actual churn cases in the holdout set, underscoring the ongoing difficulty of accurately capturing leaving customers.
Ultimately, the findings demonstrate that predictive performance can be closely matched across different algorithms. Reliable churn prediction requires a holistic approach using multiple evaluation metrics rather than relying on accuracy alone.
References
Arlot, S., & Celisse, A. (2010). A survey of cross validation procedures for model selection. Statistics Surveys, 4, 40–79. https://doi.org/10.1214/09-SS054
Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32. https://doi.org/10.1023/A:1010933404324
Burez, J., & Van den Poel, D. (2009). Handling class imbalance in customer churn prediction. Expert Systems with Applications, 36(3), 4626–4636. https://doi.org/10.1016/j.eswa.2008.05.027
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. Annals of Internal Medicine, 162(1), 55–63. https://doi.org/10.7326/M14-0697
Coussement, K., & Van den Poel, D. (2008). Churn prediction in subscription services: An application of support vector machines while comparing two parameter selection techniques. Expert Systems with Applications, 34(1), 313–327. https://doi.org/10.1016/j.eswa.2006.09.038
De, S., & Prabu, P. (2022). Predicting customer churn: A systematic literature review. Journal of Discrete Mathematical Sciences and Cryptography, 25(7), 1965–1985. https://doi.org/10.1080/09720529.2022.2133238
Edwine, N., Wang, W., Song, W., & Ssebuggwawo, D. (2022). Detecting the risk of customer churn in telecom sector: A comparative study. Mathematical Problems in Engineering, 2022, 8534739. https://doi.org/10.1155/2022/8534739
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874. https://doi.org/10.1016/j.patrec.2005.10.010
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232. https://doi.org/10.1214/aos/1013203451
He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. https://doi.org/10.1109/TKDE.2008.239
Hung, S. Y., Yen, D. C., & Wang, H. Y. (2006). Applying data mining to telecom churn management. Expert Systems with Applications, 31(3), 515–524. https://doi.org/10.1016/j.eswa.2005.09.080
IBM. (n.d.). Telco customer churn dataset. IBM customer churn prediction sample. https://github.com/IBM/telco-customer-churn-on-icp4d
Imani, M., Joudaki, M., Beikmohammadi, A., & Arabnia, H. R. (2025). Customer churn prediction: A systematic review of recent advances, trends, and challenges in machine learning and deep learning. Machine Learning and Knowledge Extraction, 7(3), 105. https://doi.org/10.3390/make7030105
Jain, H., Khunteta, A., & Sharma, N. (2026). Telecom customer churn prediction: A detailed systematic review of data, methodology, and evaluation methods. Telecommunication Systems, 89(3), 1–48. https://doi.org/10.1007/s11235-026-01491-3
Kraskov, A., Stögbauer, H., & Grassberger, P. (2004). Estimating mutual information. Physical Review E, 69(6), 066138. https://doi.org/10.1103/PhysRevE.69.066138
Larivière, B., & Van den Poel, D. (2005). Predicting customer retention and profitability by using random forests and regression forests techniques. Expert Systems with Applications, 29(2), 472–484. https://doi.org/10.1016/j.eswa.2005.04.043
Lima, E. O., Mues, C., & Baesens, B. (2009). Domain knowledge integration in data mining using decision tables: Case studies in churn prediction. Journal of the Operational Research Society, 60(8), 1096–1106. https://doi.org/10.1057/jors.2008.161
Loh, W. Y. (2011). Classification and regression trees. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 1(1), 14–23. https://doi.org/10.1002/widm.8
Mettle, H. N. A., Ayerko, E. H., & Appiahene, P. (2026). Machine learning based customer churn prediction in telecommunication industry. Applied Computational Intelligence and Soft Computing, 2026, 8496186. https://doi.org/10.1155/acis/8496186
Neslin, S. A., Gupta, S., Kamakura, W., Lu, J., & Mason, C. H. (2006). Defection detection: Measuring and understanding the predictive accuracy of customer churn models. Journal of Marketing Research, 43(2), 204–211. https://doi.org/10.1509/jmkr.43.2.204
Nguyen, V. T., Chan, J., & Bailey, J. (2014). Reconsidering mutual information based feature selection: A statistical significance view. In Proceedings of the AAAI Conference on Artificial Intelligence, 28(1). https://doi.org/10.1609/aaai.v28i1.8953
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo Wilson, E., McDonald, S., ... Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information criteria of max dependency, max relevance, and min redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226–1238. https://doi.org/10.1109/TPAMI.2005.159
Quinlan, J. R. (1993). C4.5: Programs for machine learning. Morgan Kaufmann.
Saito, T., & Rehmsmeier, M. (2015). The precision recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), e0118432. https://doi.org/10.1371/journal.pone.0118432
Sana, J. K., Abedin, M. Z., Rahman, M. S., & Rahman, M. S. (2022). A novel customer churn prediction model for the telecommunication industry using data transformation methods and feature selection. PLOS ONE, 17(12), e0278095. https://doi.org/10.1371/journal.pone.0278095
Van den Poel, D., & Larivière, B. (2004). Customer attrition analysis for financial services using proportional hazard models. European Journal of Operational Research, 157(1), 196–217. https://doi.org/10.1016/S0377-2217(03)00069-9
Verbraken, T., Verbeke, W., & Baesens, B. (2014). Profit optimizing customer churn prediction with Bayesian network classifiers. Intelligent Data Analysis, 18(1), 3–24. https://doi.org/10.3233/IDA-130625
Verbeke, W., Dejaeger, K., Martens, D., Hur, J., & Baesens, B. (2012). New insights into churn prediction in the telecommunication sector: A profit driven data mining approach. European Journal of Operational Research, 218(1), 211–229. https://doi.org/10.1016/j.ejor.2011.09.031
Verbeke, W., Martens, D., & Baesens, B. (2014). Social network analysis for customer churn prediction. Applied Soft Computing, 14, 431–446. https://doi.org/10.1016/j.asoc.2013.09.017
Verbeke, W., Martens, D., Mues, C., & Baesens, B. (2011). Building comprehensible customer churn prediction models with advanced rule induction techniques. Expert Systems with Applications, 38(3), 2354–2364. https://doi.org/10.1016/j.eswa.2010.08.023
Wagh, S. K., Andhale, A. A., Wagh, K. S., Pansare, J. R., Ambadekar, S. P., & Gawande, S. H. (2024). Customer churn prediction in telecom sector using machine learning techniques. Results in Control and Optimization, 14, 100342. https://doi.org/10.1016/j.rico.2023.100342
Wei, C. P., & Chiu, I. T. (2002). Turning telecommunications call details to churn prediction: A data mining approach. Expert Systems with Applications, 23(2), 103–112. https://doi.org/10.1016/S0957-4174(02)00030-1
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Ernest Edem Dzisah (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
