DECOMPOSITION OF MULTI-DIMENSIONAL

POVERTY INDEX:
A STUDY THROUGH
SHAPLEY MACHINE LEARNING

Jayanjana Manna1, Santosh Nandi2, Dr. Sugata Sen3

JEL code: C6

Keywords: Multi-dimensional Poverty,ShapleyValue Decomposition, SHAP

Machine Learning.

1. INTRODUCTION

Poverty is the most appealing problem of human society. For a long time
measures have been taken to eradicate this menace from the human life. But its
very exsistance proves the futility of those measures. There are voluminous
research to find the loopholes of those futile exercises. One of the reasons which
is getting greater importance about the failure of poverty eradication is said to
be the concept of poverty. Gradually a larger section of the social scientists are
accepting that poverty is a multi-dimensional concept. So, the research on
poverty is shifting towards various issues of multi-dimensional poverty. This
work wants to develop a decomposition mechanism,so that the contribution of
individual dimensions can be estimated properly. To that respect Shapley Value
Decomposition with the help of Un-supervised Machine Learning is utilized.

2. REVIEW OF LITERATURE

The idea of poverty first appeared in the thinking of Confusious nearly 500
years before the birth of Christ. Poverty also appeared in the philosophy of
Aristotol through the idea of ditributive justice. But poverty started to gain

1
Department of Computer Science and Applications, Panskura Banamali College, jayanjanamail@gmail.com
2
Department of Computer Science and Applications, Panskura Banamali College, nandisantosh@gmail.com
3
Department of Economics, Panskura Banamali College, sensugata@gmail.com

1
importance in social policy formulations from the period of Mercentailism. The
Marcentailists conceived poverty as an essential pre-condition for economic
development. But instead of viewing poverty as a social problem, it was viewed
as a necessity towards the stability of the ongoing social structure. Though the
idea of re-distributive justice was thought by Aristotol, it started to draw social
attention with the writings of Charles de Montesquieu in the 18 th century.
Following the ideas of Montesquieu states started to share the burden of poverty
with the emergence of modern capitalism. Through the coming out of modern
welfare state and the growing importance of re-distributive justice the
estimation of poverty emerged as an important activity of the state (Ravallion,
2016).

Philosophically it is accepted that the idea of poverty is related to the notion

of well being. So the assesment of poverty refers to the achievement of certain
level of well being. With time this concept of well being is transformed to the
idea of being well. This transformation opened a whole new era in poverty
research through capability approach. Poverty started to be appear as capability
deprivation. This shift in the perspectives in the poverty research gradually
accepts capability based indicators for poverty measurement. With the
emergence of capability approach the poverty also becomes multi-dimensional.
Naturally the multi-dimensional poverty (MP) index is right now the most
accepted indicator of poverty (Sen, 1976).

The idea of multi-dimensional poverty index is carried forword by the

development of axiomatic identification process by Chakravarty and
Burguignon (Bourguignon & Chakravarty, 2003). Alkire and Foster developed
an index on the basis of this identification process (Alkire & Foster, 2011). This
AF index is the logical starting point of this work.

Apart from measuring the composite effect of the multi-dimensional poverty,

a large volume of research appeared on the decomposition of composite index.
The sub-group decomposibilty of MP index became very important due to its
special importance in policy formulation.

The Shapley Value Decomposition is a solution concept in the findings of

influencial causal factors. This type of decomposition takes into consideration
the average of the marginal contributions of a factor under different
combinations. To that respect, the concerned factor is first withdrawn from the
model and the rest of the factors are permuted to form different distributions.

2
Gradually, the withdrawn factor is added to each of the combination and the
marginal contribution of the added factor in a specific distribution is counted.
The average of marginal contributions of the stated factor from all the
distributions is the influence of that very factor on the composite influence. In
this way, the average contribution of all the factors are determined. The
aggregation of all these factoral influences deliver the overall variation of the
depedent factor. In this way, the Shapley Value Method decompose the overall
variation of the composite dependent factor into the independent causal
factors(Shorrocks, 2013).

To execute the Shapley Value Decomposition of the multi-dimensional

poverty index machine learning can be used. Machine Learning(ML) is a
technique of data analytics, that instructs computer to learn from experience.
Machine Learning algorithms use computational methods to “learn” information
directly from data without depending on a pre-set equation as a model. The
algorithms gradually improve their performance as a number of specimens
accessible for learning increases(Kubat, 2017).

Supervised machine learning creates a model that makes predictions

based on evidence in the appearance of ambiguity. A supervised learning
algorithm takes a known set of input data and known outputs of the data and
trains a model to produce feasible predictions for the output of new data.
Supervised learning applies classification and regression techniques to develop
predictive models. Classification techniques under supervised learning predict
discrete outputs. Classification models classify input data into categories. Some
applications of classification include medical imaging, credit scoring, and
speech recognition. Classification techniques can be familiarised if the data are
tagged, categorized, or separated into specific groups or classes. Some common
algorithms for classification are- support vector machine
(SVM), boosted and bagged decision trees,discriminant analysis, Naïve
Bayes, k-nearest neighbour, logistic regression, and neural networks.
Regression techniques under supervised learning predict successive reactions.
Regression techniques are used the models are working with a data range or if
the nature of output is a real number. Some common regression algorithms are-
linear model, nonlinear model, boosted and bagged decision trees, neural
networks, and adaptive neuron-fuzzy learning(Chopra, 2018).

3
 Unsupervised learning reveals hidden patterns or inherent, structures in
data. It is used to draw decisions from datasets consisting of input data without
pre-set outputs. Clustering is the most common unsupervised learning
technique. It is used for experimental data analysis to reveal hidden patterns or
groupings in data. Some common applications for cluster analysis are-market
research, gene sequence analysis, and object recognition. Common algorithms
for performing clustering include k-means and k-medoids, hierarchical
clustering, Gaussian mixture models, hidden Markov models, self-organizing
maps, fuzzy c-means clustering, and subtractive clustering(Srinivasaraghavan &
Joseph, 2019).

Understanding of human learning and its computational aspect is a

pointful scientific aim of Machine Learning. Undoubtedly the key ofhuman
intelligence is theircapability of learn. Thus an overall understanding of human
learning process is very important to understand human intelligence. ML may
benefit us from the basic principles of human learning and that may lead to the
invention of more fruitful learning techniques (Theobald, 2017).

An alternative to the traditional learning methods has been provided by

Shapley Value Machine Learning. This method of machine learning can be
applied to decompose the individual influence of causal factors on multi-
dimensional poverty using Shapley Decomposition. Shapley Value Machine
Learning entails the approximation of the factoral contributions through
Shapley decomposition method. One of the framework called SHAP executes
the Shapley Machine Learning in reality. SHAP puts an importance value to
each causal factors. SHAP framework takes into consideration LIME,
DeepLIFT and layer-wise relevance propagation. But in reality finding the exact
value of SHAP is really challanging. But, those values can be approximated
through combining the current Additive Feature Attribution Methods. Help of
Max SHAP and Deep SHAP can also be taken for exact computation of SHAP
values (Lundberg & Lee, 2017).

Thus it appears that machine learning can play an Important role in

poverty analysis and sub-sequent policy formulations through machine learning
based decomposition. Apart from the development of proper measure under the
multi-dimensional framework the machine learning technology can play a
potent role to find relative importance of different influencing factors. To find
the relative importance of the explanatory dimensions Shapley Value

4
Decomposition can be applied. The decomposition method as prescribed by
shapley can easily be executed through Unsupervised Machine Learning. The
shapley value decomposition, with the use of unsupervised machine learning
can help us to find the influence of difference explanatory dimensions on the
composite level of poverty. The proper estimation of relative importance of this
factor can usher a new dawn in poverty eradication policies. So the specific
objective of the study is –

3. OBJECTIVES

To decompose the AF multi-dimensional poverty index with the help of

Shapley Value Machine Learning to estimate the relative importance of
individual dimensions.

4. FINDINGS

This work accepts he dual cut-off method as developed by Alkire &

Foster(Alkire & Foster, 2011). To that respect, this work accepts the
identification procedure developed by Chakravorty and
Burguenon(Bourguignon & Chakravarty, 2003).

Let, there are n individual, expressed as i=1,2, … n and k dimension

expressed as j=1,2,…k. Then the performance level of each individual on each
dimension can be expressed as an n×k real valued non-negative matrix. Thus,
each row vector yi={yij} when j=1,2,…k. explains the performance vector of
each individual.

Let, z is a vector of threshold values when, z={zj}.

So, on the basis of yi and z we can create gi0 for each individual, when
gi0={1,0}.

Here gij0 =1, when yij<zj and

gij0 =0, when yij≥zj

Now it ci=|gij0| i.e., the sum of the gj0 and any individuals. The vector c
will show the numbers of dimensions where each individual is lying below the
established threshold.

5
 But, ciis not sufficient to conclude about the multi-dimensional poverty of
an individual.

Let us assume that to become multi-dimensionally poor it is necessary to

be poor in d dimensions.

If, d=1 then, it will be union approach

And if d=k then, it will be intersection approach

Again, d may be 1 < d < k, so generalised statement can be

d= 1 ≤ d ≤ k

We can apply a value judgement to fix the value of d.

Then a household become multi-dimensionally poor if,

ci ≥ d

From here, we can create the head out ratio of multi-dimensionally poor – H.

Let, M is a set of m individual whose ci ≥ d,

Thus, H= m/n

Now, the rate of Poverty P = m/n×100

So, it appears that, P = f(y,z,d)

That means, the poverty rate P is influenced by k dimensions. Through

decomposing we want to decompose the total effect of P into the marginal
effects of different dimensions.

We are interested to decompose the level of poverty on the basis of

different contributory factors or dimensions. It is quite natural that, the expected
value of poverty is 0. So, the difference between expected and observed value
of P is P. To decompose this difference in poverty due to different dimensions
Shapley value decomposition is used. This method calculates the average of
marginal contributions of each dimension. In our model if we are interested to
find the contribution of Kth dimension, then we would find different

6
combination of K-1 dimensions. So, the total no of combinations without K
dimensions is –
(K-1)
C1 + (K-1)C2 + (K-1)C3 + (K-1)C4 + …. + (K-1)CK-1

Next, we would add the Kth dimension with each of the combination to
find the marginal effect of Kth dimension on that particular combination.
Naturally, we would get,

C1 + (K-1)C2 + (K-1)C3 + (K-1)C4 + …. + (K-1)CK-1 marginal combinations.

(K-1)

According to Shapley decomposition, the contribution of Kth dimension is the

average of all the marginal contributions of Kth dimension.

So, the total no of combinations is –

∑𝐾−1
𝑝=1 (𝑘 − 1)𝐶𝑝 = 𝜇

Let, the marginal contribution of Kth dimension from pth combination is

𝜑p. So, the set of marginal contribution is of Kth dimension is-

ηk= (𝝋1k, 𝝋2k, …, 𝝋µk )

So, the average of marginal contribution of

𝜇 1
𝐾 = 𝐶𝑂𝑁k = ∑𝑖=1 𝜑ik
𝜇

We have 𝜇 combinations for each dimension that means in the whole

model we have K𝜇 combinations. Each combination is nothing but some sort of
equation. These equations are formed through appropriate regression. Machine
learning procedure through artificial intelligence here will reduce the K𝜇
equations to K equations on the basis of most efficient contribution of each
dimension. This detailed operation can be executed using the SHAP framework
easily. Using the SHAP can deliver executable efficient as well as interpretable
dimensional influence.

5. CONCLUSION

Research on poverty and inequality has established that the indicators of

poverty are multi-dimensional. Identification of poor can easily be done through

7
the dual cut-off system as developed by AF method. This work establishes a
mathematical model to show that the influence of causal dimensions on
composite level of poverty can be decomposed through Shapley Value
Decomposition. This work also establishes that efficient and interpretable
dimensions specific influence levels can be determined through SHapley
Additive exPlanation Method. The estimation through Shapley Value Machine
Learning will undoubtedly open a whole new world of policy formulation for
poverty eradication.

Works Cited

Alkire, a., & Foster, J. (2011). Counting and multidimensional poverty measurement. Journal of Public
Economics , 476-487.

Bourguignon, F., & Chakravarty, S. R. (2003). The Measurement of Multidimensional Poverty. The
Journal of Economic Inequality , 25-49.

Chopra, R. (2018). Machine Learning. New Delhi: Khanna Book Publishing.

Kubat, M. (2017). An Introduction to Machine Learning. Cham: Springer.

Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model. Retrieved March 19,
2019, from NIPS Proceedings: https://computer.org/csdl/proceedings-
article/icpr/1994/00576879/12OmNx965uE

Ravallion, M. (2016). THE ECONOMICS OF POVERTY - HISTORY, MEASUREMENT AND POLICY. New
Delhi: Oxford University Press.

Sen, A. (1976). Poverty: An Ordinal Approach to Measurement . Econometrica , 219-231.

Shorrocks, A. F. (2013). Decomposition Procedures for Distributional Analysis: A Unified Framework

Based on the Shapley Value. The Journal of Economic Ineuality , 99-126.

Srinivasaraghavan, A., & Joseph, V. (2019). Machine Learning . Hoboken: Wiley.

Theobald, O. (2017). Machine Learning For Absolute Beginners: A Plain English Introduction (Second
Edition). Seattle: Scatterplot Press.

8
9