<jupyter_start><jupyter_text>Indexing/Selction/Slicing of Datahttps://i.stack.imgur.com/FzimB.pnghttps://stackoverflow.com/questions/22149584/what-does-axis-in-pandas-meanenter image description hereIn Pandas:axis=0 means along "indexes". It's a row-wise operation.Suppose, to perform concat() operation on dataframe1 & dataframe2, we will take dataframe1 & take out 1st row from dataframe1 and place into the new DF, then we take out another row from dataframe1 and put into new DF, we repeat this process until we reach to the bottom of dataframe1. Then, we do the same process for dataframe2.Basically, stacking dataframe2 on top of dataframe1 or vice a versa.E.g making a pile of books on a table or flooraxis=1 means along "columns". It's a column-wise operation.Suppose, to perform concat() operation on dataframe1 & dataframe2, we will take out the 1st complete column(a.k.a 1st series) of dataframe1 and place into new DF, then we take out the second column of dataframe1 and keep adjacent to it (sideways), we have to repeat this operation until all columns are finished. Then, we repeat the same process on dataframe2. Basically, stacking dataframe2 sideways.<jupyter_code>import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Let's create a DataFrame<jupyter_code>dates = pd.date_range('1/1/2000', periods=8)
# dates are treated as index
#  that means you can access a particular entry using that index
df = pd.DataFrame(np.random.randn(8,4), 
                 index=dates, columns=['A', 'B', 'C', 'D'])
# if we select a column it's like a list now
#  now, we can get the desired enteries by using the indexing 
#  we already have dates as index, we will use the location to see 
#  it's the same concpet as list/list of lists 
#  [[1,2,3]] = it has 1 list and that list has 3 elements inside 
s = df['A']
print(s.shape)
s[dates[5]]<jupyter_output>(8,)<jupyter_text>Inter-change the column names and values (swap)It's useful for in-place column transformation to a subset of the columns<jupyter_code>df[['A', 'B']] = df[['B', 'A']]
df<jupyter_output><empty_output><jupyter_text>Inter-change Multiple columns (swap)The correct way to swap column values is by using raw values:<jupyter_code>#  getting the subset of data/only few columns
df[['A', 'B']]
#  the column values move around
#  the vlaues A become values of B and vice versa
df.loc[:, ['B', 'A']] = df[['A', 'B']].to_numpy()
df[["A", "B"]]<jupyter_output><empty_output><jupyter_text>Attribute AccessAccessing an index on a Series or column on a DataFrame directly as an attribute<jupyter_code># let's create a series
sa = pd.Series([1,2,3], index = list('abc'))

dfa = df.copy()
sa
#  accessing the elements 
# using the index you can access the item in the series 
sa.b
#  let's select a column from our earlier df

dfa.A<jupyter_output><empty_output><jupyter_text>Assigning index to an exist column/new column as index<jupyter_code># creating anew column of the same length as index
dfa['F'] = list(range(len(df.index))) # use dfa['A'] to make a new col 'A'
# reassign an existing column as index
dfa.A = list(range(len(df.index))) # dfa.A notation is fine if the col already exists
dfa
dfa = dfa.drop(columns='F')
dfa.keys()<jupyter_output><empty_output><jupyter_text>Assigning a dict to a row of a DataFrame<jupyter_code>#  create a two col df 
X = pd.DataFrame({'x': [1,2,3], 'y':[3,4,5]})
X<jupyter_output><empty_output><jupyter_text>Changing values of a column using row indexing<jupyter_code># change the value of the 2nd row
# just X.loc[1] also works!
X.iloc[1] = {'x':99, 'y':88}
X.iloc[1]['y'] = 500
X
#  if we change the value of only 1 column, the rest of the row will get NaNs
X.iloc[2] = {'y':55} # the value of X at the 2nd location of the 1st element of the 2nd row is NaN
X
#  addign anotehr column
X['z'] = [1,2,3]
X.iloc[1] = 100
X<jupyter_output><empty_output><jupyter_text>Slicing Ranges<jupyter_code>#  s was df['A']
# s is like a series now
s
s[::] # same as s[:], double colons s[::2] are used for skipping
s[:5] # first 5 same sa s[0:5]
s[::2] # every other 
s[::-1] # in reverse order
s2 = s.copy()
# turns first five rows into 0
s2[:5] = 0
s2
df[:3] # outputs first 3 rows
df[::-1]<jupyter_output><empty_output><jupyter_text>Selection by Label<jupyter_code>dfl = pd.DataFrame(np.random.randn(5,4), 
                  columns = list('ABCD'),
                  index=pd.date_range('20130101', periods=5))

dfl
dfl.loc['20130101': '20130103']
# to access the items of a row
# dfl.loc['20130101'][0]

dfl.loc[:'20130103']<jupyter_output><empty_output><jupyter_text>More index (row) based indexing<jupyter_code>s1 = pd.Series(np.random.randn(6), index=list('abcdef'))
s1
# 'c' row onwards
s1.loc['c':]
s1.loc['c'] = 0
s1
s1.loc['c':] = 0
s1
df1 = pd.DataFrame(np.random.randn(6,4),
                  index = list('abcded'),
                  columns=list('ABDC'))
df1
df1.loc['d']<jupyter_output><empty_output><jupyter_text>Selecting particular rows and columns<jupyter_code># a,c,d rows and B and D columns
df1.loc[['a', 'c', 'd'], 'B':'D']
df1.loc['c':, 'A':'C']<jupyter_output><empty_output><jupyter_text>if the index aren't unique, you might have problems slicing the df using .locKeyError: "Cannot get left slice bound for non-unique label: 'd'"<jupyter_code>df1.loc['d':, 'A':'C']<jupyter_output><empty_output><jupyter_text>iloc, loc<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1283996299722479, 'fraction_numerical': 0.028862164662349678, 'mean_word_length': 4.960308710033076, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '524421', 'license': 'MIT', 'n_tokens_mistral': 2017, 'n_tokens_neox': 1918, 'n_words': 641, 'path': 'data_manipulation_Pandas.ipynb', 'repo_name': 'manjulamishra/DS-Code-Reproducibility', 'size': '5405'}