<jupyter_start><jupyter_text>Pandas dataframesPandas dataframes extend NumPy 2D arrays by giving labels to the columns and an explicit index, also to the rows.<jupyter_code>import numpy as np
import pandas as pd
pop2014 = pd.Series([100,99.3,95.5,93.5,92.4,84.8,84.5,78.9,74.3,72.8],
                    index=['Java','C','C++','Python','C#','PHP','JavaScript','Ruby','R','Matlab'])
pop2015 = pd.Series({'Java': 100,'C': 99.9,'C++': 99.4,'Python': 96.5,'C#': 91.3,
                     'R': 84.8,'PHP': 84.5,'JavaScript': 83.0,'Ruby': 76.2,'Matlab': 72.4})<jupyter_output><empty_output><jupyter_text>1. Making DataFrames from Series objects, Python dicts, and NumPy arrays2. Setting indexes3. Selecting, combining and creating columns4. Performing database-style operations like relational joins on dataframes From the two series, we can create a dataframe. The indexes will be matched automatically.<jupyter_code>twoyears = pd.DataFrame({'2014': pop2014, '2015': pop2015})
# keys will be used as column names
twoyears
# We can sort the dataframe using the values in one of the columns

twoyears = twoyears.sort('2015',ascending=False)
twoyears<jupyter_output><empty_output><jupyter_text>Since Pandas is built on top of NumPy, there's a NumPy array inside every DataFrame.We can extract it by asking for the attribute, "values".<jupyter_code>twoyears.values
#We can extract the index and the names of the columns

twoyears.index
twoyears.columns
#indexing the dataframe with brackets naturally gives a column

twoyears['2014']
# To select a subset of rows from the dataframe use iloc or loc

twoyears.iloc[0:2]
twoyears.loc['C':'Java']
# We can do numerical operations on entire columns and store the result in a new column

twoyears['avg'] = 0.5*(twoyears['2014'] + twoyears['2015'])

twoyears<jupyter_output><empty_output><jupyter_text>There are more ways to make a DataFrame. For instance, we could use a Python dict to specify every row as a dict item and then give Pandas DataFrame a list of such dictionaries.<jupyter_code>presidents = pd.DataFrame([{'name': '<NAME>', 'inag': 2009, 'birthyear': 1961},
                           {'name': '<NAME>', 'inag': 2001, 'birthyear': 1946},
                           {'name': '<NAME>', 'inag': 1993, 'birthyear': 1946}, 
                           {'name': '<NAME>', 'inag': 1989, 'birthyear': 1924}])
presidents
#We can choose one of the columns to be used as the index

presidents_indexes = presidents.set_index('name')
presidents_indexes
presidents_indexes.loc['<NAME>']     #gives entire record for Clinton
#Then we can choose the right column

presidents_indexes.loc['<NAME>']['inag']
#Or we could select the column first and then index the president we want

presidents_indexes['inag'] 
presidents_indexes['inag']['<NAME>']<jupyter_output><empty_output><jupyter_text>In Pandas, joins are performed with merge function<jupyter_code>presidents_fathers = pd.DataFrame([{'son': '<NAME>','father':'<NAME>, Sr.'},
                                  {'son': '<NAME>','father':'<NAME>'},
                                  {'son': '<NAME>','father':'<NAME>'}])
presidents_fathers
# we combine two tables by matching values between two columns. In this case, we combine presidents 
#and their fathers using the column, name, for the first DataFrame and the column, son, for the second.
# We specify these columns with the left_on and right_on arguments to merge.


pd.merge(presidents, presidents_fathers,left_on='name',right_on='son')
#The resulting table has a redundant column that we can drop. 
#add the Pandas command, drop. We need to specify that the 
#column to drop is son and that it's indeed a column. So the axis is one.

pd.merge(presidents, presidents_fathers,left_on='name',right_on='son').drop('son',axis=1)
#notice also that the join omitted the record for <NAME> since it could not match him to a father. 
#We can do a slightly different kind of join that would include also unmatched records.
#That would be a so-called left join

pd.merge(presidents, presidents_fathers,left_on='name',right_on='son',how='left').drop('son',axis=1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12342386032977691, 'fraction_numerical': 0.03467507274490786, 'mean_word_length': 4.635245901639344, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '407875', 'license': 'MIT', 'n_tokens_mistral': 1388, 'n_tokens_neox': 1236, 'n_words': 443, 'path': '09 dataframes.ipynb', 'repo_name': 'AryaDevops/Python-for-DataScience', 'size': '4228'}