<jupyter_start><jupyter_text>The fundamental data structures are Series and DataFrame<jupyter_code>s = pd.Series(np.random.randn(4), index =['a','f','t','y'])
s
s['a']<jupyter_output><empty_output><jupyter_text>The data argument can be dict, ndarray, or even a scalar etc<jupyter_code>data = {'a': 3, 'b':4, 'c': 5, 'f':'something_else'}
data
s_dict = pd.Series(data, index=data.keys())
s_dict<jupyter_output><empty_output><jupyter_text>When the dict doesn't have a matching key, it's not added. And when the index key is missing a value attached, NaN is given<jupyter_code>s_dict_with_nan = pd.Series(data, index = ['a','b', 'j'])
s_dict_with_nan<jupyter_output><empty_output><jupyter_text>Works differently with scalars<jupyter_code>s_scalar = pd.Series(3, index=range(5))<jupyter_output><empty_output><jupyter_text>Works the same way even if you pass a list with one element, like [3] but fails if you pass [3,4] because expects 5 and not 2<jupyter_code>s_scalar<jupyter_output><empty_output><jupyter_text>Series works just like an ndarray, if you have worked with numpy before. I do not have a lot of experience with numpy so can't comment on the full capabilites but according to what I know, you can apply vectorized operations to get a better code performance, slice in the same way we do with numpy ndarrays.<jupyter_code>s<jupyter_output><empty_output><jupyter_text>MENTIONING PYTHON DATA TYPE IN VARIABLE NAME IS NOT GOOD PRACTICE, SO      TRY NOT TO<jupyter_code>s[s>s.median()]
s[[3, 0, 1]]
np.exp(s)
s.values
s.keys
s.keys()
s.index

try :
    some_random_var = s['g']   #   Raises key error
except KeyError :
    print('Caught key Error')
    
s.f # Can also access elements this way
s.a
s<jupyter_output><empty_output><jupyter_text>Vectorized operations - Start of the end of matlab RIP<jupyter_code>s+s          
s*2
s*3
s/2<jupyter_output><empty_output><jupyter_text>Moving to Pandas DataFrame  According to definition from https://pandas.pydata.org/pandas-docs/stable/dsintro.htmldsintro __DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dict of Series objects. It is generally the most commonly used pandas object. Like Series, DataFrame accepts many different kinds of input: __- Dict of 1D ndarrays, lists, dicts, or Series- 2-D numpy.ndarray- Structured or record ndarray- A Series- Another DataFrame __Along with the data, you can optionally pass index (row labels) and columns (column labels) arguments.__<jupyter_code>data = {'one': pd.Series([1,2,3,4], index=['a','b','c','d']),
        'two': pd.Series([3,4,5,56,6], index=['a','b','f','e','y'])}
df = pd.DataFrame(data)
df<jupyter_output><empty_output><jupyter_text>As you can see that it merged the two index together and filled the rest of the values with NaN<jupyter_code>df_1 = pd.DataFrame(data, index=['a','b','c','f'])
df_1
df_2 = pd.DataFrame(data, columns=['two'])
df_2
df
df.index
df.columns
data_for_dict = { 'one': [1,2,4,5],
                  'two': [2.,5.4,4.,5]}
df_from_dict = pd.DataFrame(data_for_dict)
df_from_dict['one']
data = np.ones((2,8))
data
data.keys()
s = pd.Series([3,4,5,5])
s
s.values
s.keys()<jupyter_output><empty_output><jupyter_text>__A difference to note here is that Series does have a values and keys attribute whereas an ndarray doesn't. Good to know__<jupyter_code>rows = [[1,2,3,43],[3,5,6,6],[66,6,6]]
df_rows = pd.DataFrame(rows)
df_rows
df_rows_with_index = pd.DataFrame(rows, index=['first', 'second','third'])
df_rows_with_index    # Number of indices should always match the rows count else will raise a shape error

try:
    df_test = pd.DataFrame({'one':[2,23,4,5,56],
                        'two': [1,3,4,4]})
except ValueError as e:
    print('Value error raised')
    print(e)
df_test_2 = pd.DataFrame([[1,2,3,4],[2,3,3]])
df_test_2
df_test_3 = pd.DataFrame([[1,2,3,4],[2,3,3]], columns=['one','two','three',
                                                      'four'])
df_test_3<jupyter_output><empty_output><jupyter_text>As we can see from the above couple of test df, when we pass a dictionary as data to DataFrame, the arrays needs to be of the same length. But if we pass a row of rows, they are adjusted accordinglyStrange, but need to know the reasoning.<jupyter_code>df_test_4<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12586127698667893, 'fraction_numerical': 0.021129995406522738, 'mean_word_length': 5.151129943502825, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 50, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '668084', 'license': 'MIT', 'n_tokens_mistral': 1558, 'n_tokens_neox': 1515, 'n_words': 460, 'path': 'Untitled.ipynb', 'repo_name': 'arora-anmol/pandas-practice', 'size': '4354'}