Machine Learning-Based Approach for Exploring the Household Survey Data

The use of machine learning as a basis for making informed public policy decisions is advancing; however, its application in the area of social protection in developing societies remains under-researched. The chapter offers the novel methodology of combining an Unsupervised Machine Learning K-means clustering algorithm with descriptive statistics to explore big data collected using the household survey in order to form clusters of households. The shaped clusters represent various welfare regimes that will respond to the second research question of the study. This chapter compares density-based spatial clustering of applications with noise (DBSCAN) and K-means clustering to investigate the most efficient approach. The results revealed that K-means clustering is an efficient method to explore the survey dataset. Hence, the clusters formed using the K-means clustering approach showed common patterns relating to the insecurities associated with loss of income or property, unemployment, disasters, or disease etc. that households faced, and to the access to formal and informal social protection in each cluster. The chapter concludes that by using the K-means clustering unsupervised machine learning method, big data can be explored efficiently to target social protection interventions better, with the aim of achieving optimal results to increase the welfare of the poor.