<jupyter_start><jupyter_text>Data management with Pandas An overview of some of the data management tools in Python's [Pandas package](http://pandas.pydata.org/pandas-docs/version/0.17.1/).  Includes:* Selecting variables * Selecting observations * Indexing * Groupby * Stacking * Doubly indexed dataframes * Combining dataframes (concat) * Merging dataframesThis notebook was written by <NAME> for the NYU Stern course [Data Bootcamp](http://databootcamp.nyuecon.com/).<jupyter_code>import pandas as pd
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Reminders* Dataframes * Index and columns   Selecting variables   DatasetsWe take these examples from the data input chapter:  * Penn World Table * World Economic Outlook * UN Population DataAll of them come in an unfriendly form; our goal is to fix them.  Here we extract small subsets to work with so that we can follow all the steps.    Penn World Table This one comes with countries stacked on top of each others.<jupyter_code>data = {'countrycode': ['CHN', 'CHN', 'CHN', 'FRA', 'FRA', 'FRA', 'USA', 'USA', 'USA'],
 'pop': [1124.7939240000001, 1246.8400649999999, 1318.1701519999999, 58.183173999999994,
         60.764324999999999, 64.731126000000003, 253.33909699999998, 282.49630999999999,
         310.38394799999998],
 'rgdpe': [2611027.0, 4951485.0, 11106452.0, 1293837.0, 1752570.125, 2031723.25,
           7964788.5, 11494606.0, 13151344.0],
 'year': [1990, 2000, 2010, 1990, 2000, 2010, 1990, 2000, 2010]}
pwt = pd.DataFrame(data)
pwt
### UN Population Data<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10976388002552648, 'fraction_numerical': 0.17294192724952137, 'mean_word_length': 5.054054054054054, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '146221', 'license': 'MIT', 'n_tokens_mistral': 706, 'n_tokens_neox': 514, 'n_words': 163, 'path': 'Code/notebooks/bootcamp_data_management.ipynb', 'repo_name': 'kunal-mulki/Materials', 'size': '1572'}