<jupyter_start><jupyter_text>Computing Inequality Metrics<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

from salamanca import ineq

% matplotlib inline<jupyter_output><empty_output><jupyter_text>Measures of InequalityThe most common measure of inequality is the Gini coefficient. A less common measure is the Theil coefficient, but it has nice mathematical properties that allow it be decomposed and recomposed based on subpopulations.There is a theoretical and statistical relationship between Theil and Gini:$$t(g) = \sqrt{2} \Phi^{-1} \left( \frac{1+g}{2}\right)$$Where $\Phi$ is cumulative distribution function (CDF) of the standard normal distribution.We can compute values as follows:<jupyter_code>gini = 0.5
theil = ineq.gini_to_theil(gini)
theil<jupyter_output><empty_output><jupyter_text>Empirical work has been done which shows that Theil values from household data surveys do not match exactly with Theils as computed from theoretical distributions. The difference in values are approximately 10%.<jupyter_code>theil_emp = ineq.gini_to_theil(gini, empirical=True)
theil_emp
rel_diff = (theil - theil_emp) / theil
rel_diff<jupyter_output><empty_output><jupyter_text>DistributionsIncome is most commonly modeled as lognormal distribution (though there is gaining momentum amongst economists to use a duel-tailed Paretto distribution).<jupyter_code>dist = ineq.LogNormal(inc=1000, gini=0.4)
x = np.linspace(dist.ppf(0.1), dist.ppf(0.9), 100)
y = dist.pdf(x)
plt.plot(x, y)
plt.xlabel('Income'); plt.ylabel('Population (relative)')<jupyter_output><empty_output><jupyter_text>By default, `salamanca` assumes you wish for the distribution's income to be mean to its mean.<jupyter_code>dist.mean()<jupyter_output><empty_output><jupyter_text>However, this can be changed to assume that the income is a median income.<jupyter_code>for ismean in [True, False]:
    x = np.linspace(dist.ppf(0.1, mean=ismean), dist.ppf(0.9, mean=ismean), 100)
    y = dist.pdf(x, mean=ismean)
    label = 'mean' if ismean else 'median'
    plt.plot(x, y, label=label)
plt.xlabel('Income'); plt.ylabel('Population (relative)'); plt.legend(loc='best')<jupyter_output><empty_output><jupyter_text>Lorenz CurvesLorenz curves are a common way to view income distributions from an inequality lens.<jupyter_code>x = np.linspace(0.01, 0.999, 1000)
plt.plot(x, x, label='ideal')
for gini in [0.2, 0.6]:
    y = dist.lorenz(x, gini=gini)
    plt.plot(x, y, label=str(gini))
plt.xlabel('Income (relative share)'); plt.ylabel('Population (relative share)'); plt.legend(loc='best')<jupyter_output><empty_output><jupyter_text>We can again see the effect of applying the empirical Theil relationship on the Lorenz curves.<jupyter_code>gini = 0.5
theil = ineq.gini_to_theil(gini, empirical=True)
plt.plot(x, x, label='ideal')
plt.plot(x, dist.lorenz(x, gini=gini), label='gini')
plt.plot(x, dist.lorenz(x, theil=theil, gini=None), label='theil_emp')
plt.legend(loc='best')<jupyter_output><empty_output><jupyter_text>Populations below Income ThresholdsA common application of inequality calculations is to determine the fractions of populations in various income quantiles. A convenience function is provided to make this straightforward.<jupyter_code>dist = ineq.LogNormal(inc=1000, gini=0.4)
dist.below_threshold(500) # ~ 30% makes less than 500
dist.below_threshold(1500) # ~ 80% makes less than 500<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11120866256950541, 'fraction_numerical': 0.01990049751243781, 'mean_word_length': 6.715575620767495, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '281052', 'license': 'Apache-2.0', 'n_tokens_mistral': 1214, 'n_tokens_neox': 1148, 'n_words': 323, 'path': 'doc/notebooks/inequality.ipynb', 'repo_name': 'coroa/salamanca', 'size': '3417'}