<jupyter_start><jupyter_text>_Lambda School Data Science_ Sequence your narrativeCreate a sequence of visualizations inspired by [<NAME>'s 200 Countries, 200 Years, 4 Minutes](https://www.youtube.com/watch?v=jbkSRLYSojo).Using this [data from Gapminder](https://github.com/open-numbers/ddf--gapminder--systema_globalis/):- https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--datapoints--income_per_person_gdppercapita_ppp_inflation_adjusted--by--geo--time.csv- https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--datapoints--life_expectancy_years--by--geo--time.csv- https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--entities--geo--country.csv- https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--concepts.csv Stretch goals- [ipywidgets](https://github.com/jupyter-widgets/ipywidgets)- [Matplotlib animation](https://matplotlib.org/examples/animation/index.html)- [Connected scatter plots](http://www.thefunctionalart.com/2012/09/in-praise-of-connected-scatter-plots.html)- [Idyll markup language](https://idyll-lang.org/) for "scrollytelling" Variables --> Visual Encodings- Income --> X- Lifespan --> y- Population -->Size- Region --> Color- Year --> time / animation frame (alternative: small multiples)- Country --> annotationQualitative --> Verbal- Editorial / Contextual explanation --> audio narration (alternative: text)columns   = [ Year, Country, Region, Income, Lifespan, Population]<jupyter_code>!pip install --upgrade seaborn
# Restart runtime

import seaborn as sns
sns.__version__
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
income = pd.read_csv('https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--datapoints--income_per_person_gdppercapita_ppp_inflation_adjusted--by--geo--time.csv')


life_span = pd.read_csv('https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--datapoints--life_expectancy_years--by--geo--time.csv')
population = pd.read_csv('https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--datapoints--population_total--by--geo--time.csv')
entities = pd.read_csv('https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--entities--geo--country.csv')
concepts = pd.read_csv('https://raw.githubusercontent.com/open-numbers/ddf--gapminder--systema_globalis/master/ddf--concepts.csv')
income.shape, life_span.shape, population.shape, entities.shape, concepts.shape
income.head()
life_span.head()
population.head()
entities.head()
# Data Dictionary

concepts.head()
income.shape, life_span.shape, population.shape, entities.shape, concepts.shape<jupyter_output><empty_output><jupyter_text>**Merge DATA**<jupyter_code># Merge the Data

help(pd.merge)
df = pd.merge(income, life_span)
income.shape, life_span.shape, df.shape
df.head()

# InnerJoin: Only keeping rows with same keys, hence different number of rows
entities.head()
entities[entities.country=='abw'].T
entities.world_4region.value_counts()
entities.world_6region.value_counts()
variables = ['country', 'name', 'world_6region']

entities[variables].head()
df.head()
# Preview

pd.merge(df, entities[variables],
        how='inner', left_on='geo', right_on='country').head()
# Forgot to merge df and population earlier so

df = pd.merge(df, population)
df.head()
df = pd.merge(df, entities[variables],
             how='inner', left_on='geo', right_on='country')

df.drop(columns=['geo', 'country'], inplace=True)

# pandas rename function
df.rename(columns={
    'time': 'year',
    'income_per_person_gdppercapita_ppp_inflation_adjusted': 'income',
    'life_expectancy_years': 'lifespan',
    'populaton_total': 'population',
    'name': 'country',
    'world_6region': 'region'
}, inplace=True)

df.shape
df.head()<jupyter_output><empty_output><jupyter_text>**Explore Data**<jupyter_code>df.describe()
df.describe(exclude=[np.number])
df.country.unique()
usa = df[df.country=='United States']
usa[usa.year.isin([1818, 1918, 2018])]
china = df[df.country=='China']
china[china.year.isin([1818, 1918, 2018])]<jupyter_output><empty_output><jupyter_text>**Plot Visualization**<jupyter_code>now = df[df.year==2018]
now.head()
now.shape
sns.relplot(x='income', y='lifespan', hue='region', size='population_total', data=now);<jupyter_output><empty_output><jupyter_text>** Analyze Outliers **<jupyter_code># sns.relplot(x='income', y='lifespan', hue='region', sizes='population_total', data=now);
now[now['income'] > 80000]
# Macao is considered a country

entities[entities.name=='Macao, China'].T
qatar = now[now['country']=='Qatar']
qatar.income
qatar.income.values
qatar.income.values[0]
qatar_income = qatar.income.values[0]
qatar_lifespan = qatar.lifespan.values[0]
sns.relplot(x="income", y="lifespan", hue='region', size='population_total', data=now)

plt.text(x=qatar_income, y=qatar_lifespan, s='Qatar')
plt.title('Qatar has the highest incomes in 2018');<jupyter_output><empty_output><jupyter_text>**Plot Multiple Years**<jupyter_code>years = [1818, 1918, 2018]

centuries = df[df.year.isin(years)]

centuries.sample(5)
sns.relplot(x="income", y="lifespan", hue='region', size='population_total',
            col='year', data=centuries)

plt.xscale('log');
# Consistent number of countries per year?

centuries.groupby('year').country.count()
years_per_country = centuries.groupby('country').year.count()
years_per_country[years_per_country < 3]<jupyter_output><empty_output><jupyter_text>**Point Out Story**<jupyter_code>years = [1918, 1938, 1958, 1978, 1988, 2018]

subset = df[df.year.isin(years)]

sns.relplot(x='income', y='lifespan', hue='region', size='population_total',
           row='year', data=subset)

plt.xscale('log')
plt.xlim((150, 150000));
years
for year in years:
  

  sns.relplot(x='income', y='lifespan', hue='region', size='population_total',
            data=df[df.year==year])

  plt.xscale('log')
  plt.xlim((150, 150000));
  plt.ylim((20, 90))
  plt.title(year)
  plt.axhline(y=50, color='grey');
df[(df.year==1918) & (df.lifespan >= 50)]
df[(df.year==2018) & (df.lifespan <= 50)]<jupyter_output><empty_output><jupyter_text>**Today no countries with lifespans below 50 years** **Interactivity**<jupyter_code><jupyter_output><empty_output><jupyter_text>Assignment Exploring Gapminder **Please skip to the bottom for the plots and rationale for my plots.**<jupyter_code>df.head()
df.shape
years = [1818, 1918, 2018]

centuries_1 = df[df['year'].isin(years)]

centuries_1.head()
print("Number of unique countries:", len(centuries_1['country'].unique()))
centuries_1.shape
pd.options.display.float_format = '{:.2f}'.format
centuries_1['population_total'].describe()
df['population_total'].describe()
# The mean population totals for the year 1918 = 9709630.25

df[df['year']==1818].describe()
# df[(df.year==1918) & (df.lifespan >= 50)]
pop_above_mean_1818 = df[(df['year']==1818) & (df['population_total'] > 755085.00)]
pop_above_mean_1818.head()
pop_above_mean_1818.describe()
pop_below_mean_1818 = df[(df['year']==1818) & (df['population_total'] < 755085.00)]
pop_below_mean_1818.head()
pop_below_mean_1818.describe()
df[df['year']==1918].describe()
pop_above_mean_1918 = df[(df['year']==1918) & (df['population_total'] > 1880693.50)]
pop_above_mean_1918.head()
pop_above_mean_1918.describe()
pop_below_mean_1918 = df[(df['year']==1918) & (df['population_total'] < 1880693.50)]
pop_below_mean_1918.head()
pop_below_mean_1918.describe()<jupyter_output><empty_output><jupyter_text>**The following plots show an interesting anomaly  where  it seems larger populations tend to cluster lifespans for many regions. **The first two plots are for the year 1818, where the first plot represents regions with populations below the 1818 median, and the second plot represents regions with populations above the 1818 median.  The third and fourth plots are for the year 1918, where the first plot again represents populatoiins below the 1918 median, and the second plot represents populations above the 1919 median. Taken together, these plots lend support to the idea that larger populations require greater resources for sustainability and that might in turn affect life spans.  Hence, we see greater variability for those countries under the median for 1818. Population seems to matter less in 1919, where it seems there's little difference between high-population countries and low population countries perhaps due to the effects of Influenza Epidemic, and WWI.<jupyter_code># population below mean for 1818

sns.relplot(x='income', y='lifespan', hue='region', data=pop_below_mean_1818)

plt.axhline(y=35, color='grey');
# population above mean for 1818

sns.relplot(x='income', y='lifespan', hue='region', data=pop_above_mean_1818);

plt.axhline(y=35, color='grey');
# population below the mean in 1918

sns.relplot(x='income', y='lifespan', hue='region', data=pop_below_mean_1918);

plt.axhline(y=35, color='grey');
# population above the mean in 1918

sns.relplot(x='income', y='lifespan', hue='region', data=pop_above_mean_1918);

plt.axhline(y=35, color='grey');
"""
sns.relplot(x='income', y='lifespan', hue='region', size='population_total', data=now);

years = [1818, 1918, 2018]

centuries = df[df.year.isin(years)]

centuries.sample(5)
"""<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15530383230997247, 'fraction_numerical': 0.0382172348083845, 'mean_word_length': 6.450315457413249, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 56, 'https://': 14, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '176883', 'license': 'MIT', 'n_tokens_mistral': 3649, 'n_tokens_neox': 3184, 'n_words': 627, 'path': 'LS_DS_124_Sequence_your_narrative.ipynb', 'repo_name': 'Captmoonshot/DS-Sprint-02-Storytelling-With-Data', 'size': '9452'}