<jupyter_start><jupyter_text>Introduction In this Kernel I will go through cleaning up a dataset, establishing functions and visually displaying my results. From my dataset downloaded from Penn World Tables and other sources I plan to estimate the effect of political stability in a standard growth setting.   Import libraries<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import statsmodels.formula.api as sm<jupyter_output><empty_output><jupyter_text>The Data The data is downloaded from Penn World Tables v. 9.0. The additional measures of Government Effectiveness and Political Stability are from World Governance Indicators (WGI). The two datasets have very different structures and in order to make them compatible with eachother I have merged them using VLookup on the country codes in excel. The dataset is therefore not directly from an online source, but has received minimal work in order to make it easily accessible. Furthermore the variables 'pri' and 'sec', which measures the share of population who has completed primary education and secondary education respectively, are from the Barro-Lee dataset and is simply just copy pasted into the excel sheet.<jupyter_code>#Import data
df = pd.read_excel(r"C:\Course-in-programming\Data analysis project 1\Combined.xlsx")
df.head()<jupyter_output><empty_output><jupyter_text>We see in the head that there is a lot of missing data for Aruba. If we delve deeper into the dataset we will find that this is also the case for many other countries during the 20th century. In this project I am focusing on the last 5 years of data. Most countries are covered and we do not need to constantly remind ourselves that the financial crisis happened, even though the world was still in the aftermath.  We see more coherent data now. In order to organize this better I will take a simple mean for the 5 years by country such that we get a cross section format ease the analysis. Additionally I will abandon unnecessary variables.<jupyter_code>#Dropping 1950-2010 due to inconsistent data on majority of the variables
indexNames = df[ (df['year'] >= 1950) & (df['year'] <= 2009) ].index   
reduced_df = df
reduced_df.drop(indexNames , inplace=True)
#Renaming the variables
reduced_df.rename(columns={'gdppc':'GDPPerCapita'}, inplace=True)
reduced_df.rename(columns={'csh_i':'Investment'}, inplace=True)
reduced_df.rename(columns={'popgr':'PopulationGrowth'}, inplace=True)
reduced_df.rename(columns={'csh_g':'GovernmentExpenditure'}, inplace=True)
reduced_df.rename(columns={'gdpgr':'GDPGrowth'}, inplace=True)
reduced_df.rename(columns={'pl_i':'PPI'}, inplace=True)
#Swarm plots
def year_plot(y="Investment"): 
    
    fig = plt.figure(figsize=(9,3))
    fig1 = fig.add_subplot(1,1,1)
    fig1 = sns.swarmplot(x ="year", y = y, data=reduced_df)
    if y == "GovernmentExpenditure": 
        fig1.set_title("Figure 1: Government expenditure")
        fig1.set_ylabel("Secondary Education")
    elif y == "Investment": 
        fig1.set_title("Figure 2: Investment")
        fig1.set_ylabel("Investment")
    elif y == "PoliticalStability": 
        fig1.set_title("Figure 3: PoliticalStability")
        fig1.set_ylabel("Political Stability")
    fig1.set_xlabel("year")
    

year_plot("GovernmentExpenditure")
year_plot()
year_plot("PoliticalStability")
#Calculating means for 2010-2014
means = reduced_df.groupby('country')['GDPPerCapita', 'PopulationGrowth', 'Investment', 'GovernmentExpenditure', 'PPI', 'GDPGrowth', 'pri', 'sec', 'GovernmentEffectiveness', 'PoliticalStability'].mean()
means.head()<jupyter_output><empty_output><jupyter_text>Given the swarm plots we see that there was two large outliers. One for investments and one for Government expenditure. Let's check them:<jupyter_code>means[means['Investment']==means['Investment'].max()]
means[means['GovernmentExpenditure']==means['GovernmentExpenditure'].max()]
#Dropping the large variables, as they're unnaturally high.
means = means.drop("Turks and Caicos Islands", axis=0)
means = means.drop('Cayman Islands', axis = 0)<jupyter_output><empty_output><jupyter_text>Analysis We want to analyse the effect of political stability on growth factors. The index from WGI is based on expert opinions and surveydata and is normalised with a mean of 0. Political Stability and Absence of Violence/Terrorism measures perceptions of the likelihood of political instability and/or politically motivated violence. This includes protests and riots, civil war and terrorism. Political instability can reduce incentives to invest given the increase in risk of doing so. It is therefore interesting to see what effect political stability has on investments and education.<jupyter_code>#Jointplot
sns.jointplot(x="PoliticalStability", y="Investment", data=means, kind="reg")
#Working around position of plt.title
plt.subplots_adjust(top=0.9)
plt.suptitle('Figure 4: Political stability on investments', fontsize = 12)
plt.show()
sns.jointplot(x="PoliticalStability", y="sec", data=means, kind="reg")
plt.subplots_adjust(top=0.9)
plt.suptitle('Figure 5: Political stability on secondary education', fontsize = 12)
plt.show()<jupyter_output><empty_output><jupyter_text>There's a clear positive relationship between political stability and investments/education.  Let us see how standard economic growth measures correlate with the investment rate. Investment has been shown to be correlated to economic growth by a sheer amount of studies.<jupyter_code>#Defining the title for our correlation matrices
def title_number(x):
    title=f"Figure {x}: Correlation of growth factors"
    return title
#Correlation Matrix
def corr_matrix(data):
    corr = data[['GDPPerCapita', 'PopulationGrowth', 'Investment', 'sec', 'pri', 'GDPGrowth', 'PoliticalStability']].corr()
    sns.heatmap(corr, xticklabels=corr.columns.values, yticklabels=corr.columns.values, annot=True, fmt='.2f')
    plt.show()

plt.title(title_number(6))
corr_matrix(means)<jupyter_output><empty_output><jupyter_text>As suspected from the plots, we see a large correlation between political stability and eduction and investment. This may suggest that Political Stability influences these growth factors.  I want to highlight already established findings. Several articles such as Mankiw (1992) has showed that human capital is a determinant in growth. In addition government expenditure (Barro (1991)) and investments (Levine and Renelt (1992)) has been show to be determinants for growth. A model regressing these factors on growth should confirm previous work.  Regarding primary education as a proxy for human capital: Most countries have reached the upper limit of this variable and it is therefore excluded, as the variance in primary education could be caused by exogoneous factors.<jupyter_code>#OLS
result_basic = sm.ols(formula="GDPGrowth ~ PoliticalStability + GovernmentExpenditure + Investment + sec ", data=means).fit()
print(result_basic.summary())<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:              GDPGrowth   R-squared:                       0.284
Model:                            OLS   Adj. R-squared:                  0.255
Method:                 Least Squares   F-statistic:                     9.893
Date:                Fri, 05 Apr 2019   Prob (F-statistic):           8.73e-07
Time:                        17:17:02   Log-Likelihood:                 207.56
No. Observations:                 105   AIC:                            -405.1
Df Residuals:                     100   BIC:                            -391.9
Df Model:                           4                                         
Covariance Type:            nonrobust                                         
=========================================================================================
                            coef    std e[...]<jupyter_text>First of all we see that there is a suspicion of multicollinearity and therefore we have to be careful to interpret the significance of the results. That said, investments seems to be largely insignificant which may be explained by outliers as mentioned later in the article. Under the assumption that outliers affects the significance we estimate the effects of political stability, secondary education and government expenditure on the investment rate. These factors should affect investment as argued above.<jupyter_code>#OLS
result_basic = sm.ols(formula="Investment ~  sec + PoliticalStability + GovernmentExpenditure", data=means).fit()
print(result_basic.summary())
#OLS
result_investment = sm.ols(formula = "sec ~ Investment  + PoliticalStability + GovernmentExpenditure", data=means).fit()
print(result_investment.summary())<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:                    sec   R-squared:                       0.401
Model:                            OLS   Adj. R-squared:                  0.383
Method:                 Least Squares   F-statistic:                     22.51
Date:                Fri, 05 Apr 2019   Prob (F-statistic):           3.07e-11
Time:                        17:17:02   Log-Likelihood:                -454.09
No. Observations:                 105   AIC:                             916.2
Df Residuals:                     101   BIC:                             926.8
Df Model:                           3                                         
Covariance Type:            nonrobust                                         
=========================================================================================
                            coef    std e[...]<jupyter_text>We see that political stability is positively correlated with investments and human capital. Government expenditure seems to be slightly insignifcant at a 10 pct. level when estimating secondary education. We see that investment seems to be uncorrelated with economic growth - which is odd. There is most likely external factors that distorts the relationship. In order to investigate the relationship with a better approach I will exclude the OPEC nations from the analysis, as the growth determined by oil producing countries differ from the rest of the sample and could be regarded as outliers.<jupyter_code>#Dropping OPEC nations
means_without_oil = means.drop(['Algeria', 'Indonesia', 'Iran', 'Iraq', 'Kuwait', 'Venezuela', 'Ecuador', 'Congo, D.R.'])
#Gabon, Nigeria, Oman and <NAME> are already dropped from dropping empty variables
#Correlation matrix
plt.title(title_number(7))
corr_matrix(means_without_oil)<jupyter_output><empty_output><jupyter_text>We estimate once more.<jupyter_code>result_basic_without_oil = sm.ols(formula="GDPGrowth ~  Investment + sec + PoliticalStability + GovernmentExpenditure", data=means_without_oil).fit()
print(result_basic_without_oil.summary())
result_basic_without_oil = sm.ols(formula="Investment ~  sec + PoliticalStability + GovernmentExpenditure", data=means_without_oil).fit()
print(result_basic_without_oil.summary())
result_basic_without_oil = sm.ols(formula="sec ~  Investment + PoliticalStability + GovernmentExpenditure", data=means_without_oil).fit()
print(result_basic_without_oil.summary())<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:                    sec   R-squared:                       0.450
Model:                            OLS   Adj. R-squared:                  0.432
Method:                 Least Squares   F-statistic:                     25.37
Date:                Fri, 05 Apr 2019   Prob (F-statistic):           4.45e-12
Time:                        17:17:03   Log-Likelihood:                -416.65
No. Observations:                  97   AIC:                             841.3
Df Residuals:                      93   BIC:                             851.6
Df Model:                           3                                         
Covariance Type:            nonrobust                                         
=========================================================================================
                            coef    std e[...]
{'fraction_non_alphanumeric': 0.11416814587703727, 'fraction_numerical': 0.017992577053412943, 'mean_word_length': 3.2463172319287428, 'pattern_counts': {'":': 3, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '659320', 'license': 'MIT', 'n_tokens_mistral': 3459, 'n_tokens_neox': 3054, 'n_words': 1144, 'path': 'dataproject/Hand-in.ipynb', 'repo_name': 'NumEconCopenhagen/projects-2019-aloise3', 'size': '12400'}