<jupyter_start><jupyter_text>https://gist.githubusercontent.com/anilkumar4274/e89d8808f4f19f49f41cfc08e7fc841b/raw/8a54556d283773673457a27f643a58636702866d/titanic.csv<jupyter_code>import os
import pandas as pd
url="https://gist.githubusercontent.com/anilkumar4274/e89d8808f4f19f49f41cfc08e7fc841b/raw/8a54556d283773673457a27f643a58636702866d/titanic.csv"
titanic_train = pd.read_csv(url,sep='\t')<jupyter_output><empty_output><jupyter_text>1. Use .head() / .tail() to show the first/last 5 rows [by default]:   [.head() / .tail() ]<jupyter_code>titanic_train.head()
titanic_train.tail()
titanic_train.head(10)
titanic_train.tail(10)<jupyter_output><empty_output><jupyter_text>2. To know the dimensions of the dataset:  [ .shape ]<jupyter_code>titanic_train.shape              #156 rows x 12 columns<jupyter_output><empty_output><jupyter_text>3. To know the data types of the columns:   [ .dtypes ]<jupyter_code>titanic_train.dtypes<jupyter_output><empty_output><jupyter_text>4. To get the stats of the columns:   [ .describe() ]<jupyter_code>titanic_train.describe()<jupyter_output><empty_output><jupyter_text>5. To separate the columns based on the data type:<jupyter_code>#Here, we will separate the object data types:

categorical = titanic_train.dtypes[titanic_train.dtypes == "object"].index
categorical
titanic_train[categorical].describe()
type(titanic_train[categorical])
type(categorical)<jupyter_output><empty_output><jupyter_text>6. To delete a column from a dataset:   [ del ]<jupyter_code>titanic_train.head()
del titanic_train["PassengerId"]
titanic_train.head()<jupyter_output><empty_output><jupyter_text>7. To sort the column names:   [ sorted ]  Sorted method for all rows of the specified column:<jupyter_code>sorted(titanic_train["Name"])<jupyter_output><empty_output><jupyter_text>Sorted method for specific rows:<jupyter_code>sorted(titanic_train["Name"])[0:5]<jupyter_output><empty_output><jupyter_text>8. Indexing and Slicing:   Column wise:<jupyter_code>titanic_train["Name"][4]
titanic_train["Ticket"][5:9]<jupyter_output><empty_output><jupyter_text>Row-wise:<jupyter_code>titanic_train.loc[4]
titanic_train.loc[5:9]<jupyter_output><empty_output><jupyter_text>9. Categorising the data:   Q1. Categorise the data into survivors and non-survivors:<jupyter_code>sur = titanic_train["Survived"]
sur.describe()
new = pd.Categorical(sur)
new.describe()
type(new)
type(new.describe())<jupyter_output><empty_output><jupyter_text>rename the categories if the user wishes to  [ .rename_categories() ]<jupyter_code>rename_new = new.rename_categories(["Died","Survived"])
rename_new.describe()<jupyter_output><empty_output><jupyter_text>Q2. Categorise the data into pclass:<jupyter_code>pcls = titanic_train["Pclass"]
pcls.describe()
new = pd.Categorical(pcls)
new.describe()
rename_pcls = new.rename_categories(["Pclass1","Pclass2","Pclass3"])
rename_pcls.describe()<jupyter_output><empty_output><jupyter_text>Q3. Categorise the whole dataset:<jupyter_code>whole = titanic_train
whole.describe()
new = pd.Categorical(whole)
new.describe()<jupyter_output><empty_output><jupyter_text>10. To create a new column in the dataset :<jupyter_code>titanic_train["Fam"] = titanic_train["SibSp"] + titanic_train["Parch"]
titanic_train.head()<jupyter_output><empty_output><jupyter_text>11. To create a series using pandas built-in-func:   [ pd.Series() ]<jupyter_code>dummy = pd.Series([1,None,3,None,None,6,7,8])
dummy<jupyter_output><empty_output><jupyter_text>NaN stands for null value  12. To detect NaN (null values):   [ .isnull() ]<jupyter_code>dummy
dummy.isnull()<jupyter_output><empty_output><jupyter_text>Techniques to count/view NaN:   Q1. List the names of those people who didn't enter their age?<jupyter_code>titanic_train["Age"].isnull()
import numpy as np
missing_age = np.where(titanic_train["Age"].isnull() == True)
missing_age<jupyter_output><empty_output><jupyter_text>The array listed above has individual elements showcasing the index of the missing values/NaN  [ .loc ]<jupyter_code>titanic_train.loc[missing_age]<jupyter_output><empty_output><jupyter_text>[ len() ]  Can be used to find the frequency of the NaN values<jupyter_code>len(missing_age)                    #The array considers the whole set as 1
len(missing_age[0])                 #This is the correct way of indexing to get the frequency of NaN values<jupyter_output><empty_output><jupyter_text>13. More on np.where() techinique:   Q1. Find the name of the person who purchase the ticket at the max price:<jupyter_code>ans = np.where(titanic_train["Fare"] == max(titanic_train["Fare"]))
ans
titanic_train.loc[ans]<jupyter_output><empty_output><jupyter_text>14. To create your own dataframe: Index = row -> Axis = 0Columns = column -> Axis = 1<jupyter_code>mat = np.matrix("22,66,140;42,70,148;30,62,125;35,68,160")
mat
row_label = ['A','B','C','D']
col_head = ['Age','Height','Weight']
df = pd.DataFrame(data = mat, index = row_label, columns = col_head)
df
mat = np.matrix("22,66,140;42,70,148;30,62,125;35,68,160")

mat

row_label = ['A','B','C','D']

col_head = ['Age','Height','Weight']

df = pd.DataFrame(data = mat, index = row_label, columns = col_head)

df<jupyter_output><empty_output><jupyter_text>15. To get unique entities from a column:   [ .unique() ]<jupyter_code>titanic_train["Cabin"].unique()<jupyter_output><empty_output><jupyter_text>16. Converting one data type to another:   [ astype() ] {From numpy}<jupyter_code>chair_cabin = titanic_train["Cabin"].astype(str)
chair_cabin
for i in chair_cabin:
    print(i[0]," ")
new_Cabin = np.array([cabin[0] for cabin in chair_cabin])  #Used the concept of list comprehension
new_Cabin.ndim
new_Cabin<jupyter_output><empty_output><jupyter_text>17. Categorise the cabin column as shown:<jupyter_code>new_Cabin = pd.Categorical(new_Cabin)
new_Cabin
new_Cabin.describe()<jupyter_output><empty_output><jupyter_text>18. Other example: Categorising the name-><jupyter_code>tname = np.array([i[0] for i in titanic_train["Name"]])
tname
t = pd.Categorical(tname)
t
t.describe()
df2 = titanic_train.dtypes[titanic_train.dtypes=="object"].index
df2
titanic_train[df2].describe()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13280234070221067, 'fraction_numerical': 0.03868660598179454, 'mean_word_length': 7.943313953488372, 'pattern_counts': {'":': 0, '<': 109, '<?xml version=': 0, '>': 112, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '66409', 'license': 'MIT', 'n_tokens_mistral': 2523, 'n_tokens_neox': 2360, 'n_words': 411, 'path': 'Data-Science-HYD-2k19/Topic-Wise/PANDAS/[Practice 1] TITANIC DATASET .ipynb', 'repo_name': 'Sanjay9921/Python', 'size': '6152'}