<jupyter_start><jupyter_text>PANDAS Pandas is an open-source Python Library providing high-performance data manipulation and analysis tool using its powerful data structures. The name Pandas is derived from the word Panel Data – an Econometrics from Multidimensional data.  Key Features of Pandas    Fast and efficient DataFrame object with default and customized indexing.    Tools for loading data into in-memory data objects from different file formats.    Data alignment and integrated handling of missing data.    Reshaping and pivoting of date sets.    Label-based slicing, indexing and subsetting of large data sets.    Columns from a data structure can be deleted or inserted.    Group by data for aggregation and transformations.    High performance merging and joining of data.    Time Series functionality.  Pandas deals with the following three data structures −    Series    DataFrame    Panel  Series1D labeled homogeneous array, sizeimmutable.Series is a one-dimensional array like structure with homogeneous data.Key Points:    Homogeneous data        Size Immutable        Values of Data Mutable  Data FrameKey Points:    Heterogeneous data        Size Mutable        Data Mutable  PanelPanel is a three-dimensional data structure with heterogeneous data.It is hard to represent the panel in graphical representation.But a panel can be illustrated as a container of DataFrame.Key Points:    Heterogeneous data        Size Mutable        Data Mutable  Python Pandas - Series<jupyter_code>import pandas as pd
#pandas.Series( data, index, dtype, copy)<jupyter_output><empty_output><jupyter_text>A series can be created using various inputs like −    Array    Dict    Scalar value or constant  Create an Empty Series<jupyter_code>#import the pandas library and aliasing as pd
import pandas as pd
s = pd.Series()
print(s)<jupyter_output>Series([], dtype: float64)<jupyter_text>Create a Series from ndarrayIf data is an ndarray, then index passed must be of the same length. If no index is passed, then by default index will be range(n) where n is array length, i.e., [0,1,2,3…. range(len(array))-1].<jupyter_code>#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data1=['a','b','c','d','e']
data = np.array(['a','b','c','d','e','f'])
print(data,data.shape)
print(data1)

s = pd.Series(data)
print(s)

#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data = np.array(['a','b','c','d'])
s = pd.Series(data,index=[9,10,11,12])
print(s)
print(s[10])
#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data = {'a' : 0., 'b' : 1., 'c' : 2.}
s = pd.Series(data)
print(s)

aa=np.array(data)
print(aa)
#import the pandas library and aliasing as pd
import pandas as pd
import numpy as np
data = {'a' : 0., 'b' : 1., 'c' : 2.}
s = pd.Series(data,index=['b','c','d','a',9])
print(s)
#import the pandas library and aliasing as pd

#list1=[12,23,56]
import pandas as pd
import numpy as np
s = pd.Series([12,13,15], index=[0, 1, 2])
print(s)<jupyter_output>0    12
1    13
2    15
dtype: int64<jupyter_text>Accessing Data from Series with Position<jupyter_code>import pandas as pd

s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])
print(s)
#retrieve the first element
print(s[0])
#print(s)
print(s['c'])
print(s[2])
print(s[3])
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve the first three element
print (s[::-1])
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve the last three element
print (s[-3:])
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve a single element
print (s['a'])
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve multiple elements
print (s[['a','c','d']])
import pandas as pd
s = pd.Series([1,2,3,4,5],index = ['a','b','c','d','e'])

#retrieve multiple elements
print(s['a'])<jupyter_output>1<jupyter_text>DATA Frame Features of DataFrame    Potentially columns are of different types    Size – Mutable    Labeled axes (rows and columns)    Can Perform Arithmetic operations on rows and columns<jupyter_code>import pandas as pd
data=pd.read_csv('C:/Users/guruc/Documents/AIML/dataset/diabetes_with_names.csv')
data.head()
data.tail()
data.info()
data.dtypes
data.describe()
data.shape
data.boxplot(['Pregnancies'])
data.boxplot(figsize=(14,8))
data.plot.box()
import matplotlib.pyplot as plt

plt.boxplot(data['Pregnancies'])
data.info()
data['Pregnancies']=data['Pregnancies'].astype('category')
data['Outcome']=data['Outcome'].astype('category')
data['BMI']=data['BMI'].astype('int64')
data.info()
data.head()
data.tail()
data.shape
data.describe()
pd.isna(data)
data1=pd.read_csv('C:/Users/guruc/Documents/AIML/dataset/diabetes_with_names.csv')
data1.info()
pd.isna(data1).sum()
data.notnull().sum()
data.isna().sum()
data1.isna().sum().sum()
data1.head(10)
data1.fillna(10).head(10)
import pandas as pd
import numpy as np
#dff = pd.DataFrame(np.random.randn(10, 3), columns=list('ABC'))

dff=pd.DataFrame(np.random.randn(10,3),columns=list('ABC'))<jupyter_output><empty_output><jupyter_text>Missing Value in pandasNAN-not a numberNAT- not a time stamp/ date<jupyter_code>dff
np.nan
dff.iloc[3:5, 0] = np.nan
dff
dff.iloc[4:6, 1] = np.nan
dff
dff.isna()
dff.notna()
dff.notnull()
dff.nunique()
dff.nlargest(10,'C')
dff.nsmallest(10,'A')
dff.ndim
dff.ne
dff.fillna(100)
df1=dff.iloc[8,:2]
print(df1.shape)
print(df1)
dff.loc[:,'C':'A']
dff.iloc[5:8, 2] = np.nan

dff
dff.fillna(dff.mean())
dff.fillna(dff.mean()['B':'C'])
dff.where(pd.notna(dff), dff.mean(), axis='columns')
np.random.seed(2)
ser = pd.Series(np.arange(1, 10.1, .25)**2 + np.random.randn(37))
bad = np.array([4, 13, 14, 15, 16, 17, 18, 20, 29])
ser[bad] = np.nan
ser[bad]
methods = ['linear', 'quadratic', 'cubic']


df = pd.DataFrame({m: ser.interpolate(method=m) for m in methods})


df
ser
df.plot()<jupyter_output><empty_output><jupyter_text>data frame<jupyter_code>import pandas as pd
data=pd.read_csv('/home/govinth/Documents/data/diabetes with names.csv')
data
data.head(1)
data.tail(2)
data.shape
data.info()
data.size
data.isnull()
data.isna()
data.notna()
data.describe()
data.describe().transpose()
data.corr()
data.hist()
import numpy as np
df = pd.DataFrame(np.random.randn(5, 3), index=['a', 'c', 'e', 'f', 'h'],columns=['one', 'two', 'three'])
df
df['four'] = 'bar'
import pandas as pd
df['five'] = df['one'] > 0
df
df2 = df.reindex(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h'])
df2
pd.isna(df2['one']).sum()
df2.isna().sum()
df2.notna().sum()
df2.notnull().sum()
df2.sum()
df2.mean()
df2
df3=df2.copy()
df3
df4=df3.fillna(100)
df4
df4.isna().sum()
df3
df3.fillna(method='pad')
df3
df3.iloc[:,:2]
df3
df3['timestamp'] = pd.Timestamp('20120101')
df3
dff = pd.DataFrame(np.random.randn(10, 3), columns=list('ABC'))
dff.iloc[3:5, 0] = np.nan
dff.iloc[4:6, 1] = np.nan
dff.iloc[5:8, 2] = np.nan

dff
dff.fillna(dff.mean())
dff.mean()
dff.fillna(dff.mean()['B':'C'])
dff.mean()['B':'C']
dff.where(pd.notna(dff), dff.mean(), axis='columns')
dff.mean()
data.head()
x=data.iloc[:,:-1]
x.head()
y=data.iloc[:,-1:]
y.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17089748924954917, 'fraction_numerical': 0.029685115827437925, 'mean_word_length': 5.275021758050479, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '426192', 'license': 'MIT', 'n_tokens_mistral': 2835, 'n_tokens_neox': 2691, 'n_words': 738, 'path': 'Day14-Day15/JuputerNotebook/PandasFoundation .ipynb', 'repo_name': 'amitsagargowda/AIML-Training-Content', 'size': '7209'}