<jupyter_start><jupyter_text>Dataset DescriptionsThis notebook contains most of the datasets used in Pandas Cookbook along with the names, types, descriptions and some summary statistics of each column. This is not an exhaustive list as several datasets used in the book are quite small and are explained with enough detail in the book itself. The datasets presented here are the prominent ones that appear most frequently throughout the book.  Datasets in order of appearance* [Movie](Movie-Dataset)* [College](College-Dataset)* [Employee](Employee-Dataset)* [Flights](Flights-Dataset)* [Chinook Database](Chinook-Database)* [Crime](Crime-Dataset)* [Meetup Groups](Meetup-Groups-Dataset)* [Diamonds](Diamonds-Dataset)<jupyter_code>import pandas as pd
pd.options.display.max_columns = 80<jupyter_output><empty_output><jupyter_text>Movie Dataset Brief Overview28 columns from 4,916 movies scraped from the popular website IMDB. Each row contains information on a single movie dating back to 1916 to 2015. Actor and director facebook likes should be constant for all instances across all movies. For instance, <NAME> should have the same number of facebook likes regardless of which movie he is in. Since each movie was not scraped at the same exact time, there are some inconsistencies in these counts. The dataset **movie_altered.csv** is a much cleaner version of this dataset.<jupyter_code>movie = pd.read_csv('data/movie.csv')
movie.head()
movie.shape
pd.read_csv('data/descriptions/movie_decsription.csv', index_col='Column Name')<jupyter_output><empty_output><jupyter_text>College Dataset Brief OverviewUS department of education data on 7,535 colleges. Only a sample of the total number of columns available were used in this dataset. Visit [the website](https://collegescorecard.ed.gov/data/) for more info. Data was pulled in January, 2017.<jupyter_code>college = pd.read_csv('data/college.csv')
college.head()
college.shape
pd.read_csv('data/descriptions/college_decsription.csv')<jupyter_output><empty_output><jupyter_text>Employee Dataset Brief OverviewThe city of Houston provides information on all its employees to the public. This is a random sample of 2,000 employees with a few of the more interesting columns. For more on [open Houston data visit their website](http://data.houstontx.gov/). Data was pulled in December, 2016.<jupyter_code>employee = pd.read_csv('data/employee.csv')
employee.head()
employee.shape
pd.read_csv('data/descriptions/employee_description.csv')<jupyter_output><empty_output><jupyter_text>Flights Dataset Brief OverviewA random sample of three percent of the US domestic flights originating from the ten busiest airports. Data is from the U.S. Department of Transportation's (DOT) Bureau of Transportation Statistics. [See here for more info](https://www.kaggle.com/usdot/flight-delays).<jupyter_code>flights = pd.read_csv('data/flights.csv')
flights.head()
flights.shape
pd.read_csv('data/descriptions/flights_description.csv')<jupyter_output><empty_output><jupyter_text>Airline Codes<jupyter_code>pd.read_csv('data/descriptions/airlines.csv')<jupyter_output><empty_output><jupyter_text>Airport codes<jupyter_code>pd.read_csv('data/descriptions/airports.csv').head()<jupyter_output><empty_output><jupyter_text>Chinook Database Brief OverviewThis is a sample database of a music store provided by SQLite with 11 tables. The table description image is an excellent way to get familiar with the database. [Visit the sqlite website](http://www.sqlitetutorial.net/sqlite-sample-database/) for more detail.![data/descriptions/data/descriptions/ch09_05_erd.png](data/descriptions/ch09_05_erd.png)  Crime Dataset Brief OverviewAll crime and traffic accidents for the city of Denver from January to September of 2017. This dataset is stored in special binary form called *hdf5*. Pandas uses the PyTables library to help read the data into a DataFrame. [Read the documentation](http://pandas.pydata.org/pandas-docs/stable/io.htmlio-hdf5) for more info on hdf5 formatted data.<jupyter_code>crime = pd.read_hdf('data/crime.h5')
crime.head()
crime.shape
pd.read_csv('data/descriptions/crime_description.csv')<jupyter_output><empty_output><jupyter_text>Meetup Groups Dataset Brief OverviewData was collected through the [meetup.com API](https://www.meetup.com/meetup_api/) on five Houston-area data science meetup groups. Each row represents a member joining a particular group.<jupyter_code>meetup = pd.read_csv('data/meetup_groups.csv')
meetup.head()
meetup.shape
pd.read_csv('data/descriptions/meetup_description.csv')<jupyter_output><empty_output><jupyter_text>Diamonds Dataset Brief OverviewQuality, size and price of nearly 54,000 diamonds scraped from the [Diamond Search Engine](http://www.diamondse.info/) by <NAME>. [Visit blue nile](https://www.bluenile.com/ca/education/diamonds?track=SideNav) for a beginners guide to diamonds.<jupyter_code>diamonds = pd.read_csv('data/diamonds.csv')
diamonds.head()
diamonds.shape
pd.read_csv('data/descriptions/diamonds_description.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0925302159698831, 'fraction_numerical': 0.01089756290865861, 'mean_word_length': 6.636913767019667, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 4, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '14072', 'license': 'MIT', 'n_tokens_mistral': 1581, 'n_tokens_neox': 1464, 'n_words': 469, 'path': 'dataset_descriptions.ipynb', 'repo_name': 'PyarakaSrikanth/pandas_cookbook', 'size': '5060'}