<jupyter_start><jupyter_text>Panda SeriesA 1-dimension array of data<jupyter_code># create a Series, with a name (which will be used as Column name in a DataFrame)
# note that the data is a mix of types
pd.Series(['<NAME>', 23, 180], name='User')<jupyter_output><empty_output><jupyter_text>http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.Series.html<jupyter_code># assign to a variable (and not show output)
s = pd.Series(['<NAME>', 23, 180], name='User')
# define the index in creating the Series
# index needs to be of the same length as the data
s = pd.Series(['<NAME>', 23, 180], name='User', index=['a', 'b', 'c'])
s<jupyter_output><empty_output><jupyter_text>Series Attributes<jupyter_code>s.values # list the values in the data
s.index # index type and range
s.dtype # type of data
s.ndim # nb of dimensions
s.shape # shape of data (cols, rows)
s.size # total nb of elements, including nulls
s.is_unique # are the values in the series unique?<jupyter_output><empty_output><jupyter_text>Accessing dataData is accessed like in a standard Python list (array)<jupyter_code>s
s.index[1]
s.values[2]
# also, for multiple values at once:
s.index[[0, 2]]
s.values[[0, 2]] # pass an array on value indices to lookup
s.values[1:3] # pass an array slice<jupyter_output><empty_output><jupyter_text>Methods<jupyter_code>s = pd.Series([6, 7, 7, 8, 1, 2, 3, 4, 5, 6, 7, 7, 8, 9, 10, 2, 3, 3, 6, 4, 7, 6, 5])
s.head() # first elements (default to 5)
s.head(3) # first 3 element
s.tail() # last 5 elements
s.tail(3) # last 3 elements
s.sort_values() # sort the values
s # note this is not done in place
# need to reassign the output to s
s = s.sort_values()
s
s.sort_index() # sort the index values
s # not done in place
# We can also use the parameter `inplace=True`
s.sort_index(inplace=True)
s
s.count() # nb of elements
s.sum() # sum of all elements
s.product() # product of all elements
s.min() # mininum: equivalent to np.min(b)
s.max() # maximum: equivalent to np.max(b)
s.mean() # mean (average) of all elements
s.std() # standard deviation: equivalent to np.std(b)
s.median() # median value, i.e. the value in the middle of the sorted array
s.mode() # most frequent value...
# ...or values if multiple values have the same highest frequency:
s = pd.Series([1,1,1, 2,2,2, 3,3,3, 4,5,6])
s.mode()
s.value_counts() # the frequency distribution of all values
s.describe() # summary of the series, with stats and percentiles
s.idxmin() # index of the min value
s.idxmax() # index of the max value
s.apply(lambda x: x + 1) # apply a function to each element of the Series
s.map({1: 'one', 2: 'two', 3: 'three', 4: 'four', 5: 'five', 6: 'six', 7: 'seven', 8: 'eight'}) # map the values to another, 
# could be a Series or function
s.map(lambda x: str(x) + "%").head()
s.nunique() # number of unique values 
# does not count NaNs
s.map({1: 'one', 2: 'two', 3: 'three', 4: 'four', 5: 'five', 6: 'six', 7: 'seven', 8: 'eight'}).nunique()<jupyter_output><empty_output><jupyter_text>Conversion to Python classes<jupyter_code>s.to_dict() # convert to a python dictionary
# equivalent to 
dict(s)
s.to_json() # convert to a JSON string (JavaScript Object Notation)
list(s) # convert to a python list<jupyter_output><empty_output><jupyter_text>Data type (dtype) conversions<jupyter_code>s = pd.Series(['0', '1', 2, None])
s
s = s.astype('int8') # will raise an error because we have None in the data
s = s.astype('int8', errors='ignore') # ignore errors BUT DOES NOT CONVERT
s[0] # is still a string type
s = s.astype('string') # cast all as strings is possible, but numbers fail
s
s[3]<jupyter_output><empty_output><jupyter_text>Replacing bad valuesOften times, the data is not clean, and there are nulls (None / np.nan) or values that need conversion.To convert to a certain type, data needs to be cleaned up<jupyter_code>s = pd.Series(['0', '1', 2, None])
s = s.fillna(0) # replace None or NaNs with the given value
s
s.astype('int8')<jupyter_output><empty_output><jupyter_text>Remove bad valuesWe can remove the bad values entirely<jupyter_code>s = pd.Series(['0', '1', 2, None, None, None])
s
s.dropna()<jupyter_output><empty_output><jupyter_text>Backfilling bad valuesSometimes it is necessary to back fill with known values<jupyter_code>s = pd.Series([1,2,3, None, None, None, 4,5,6])
s.values
s.fillna(method='ffill') # forward fill with last valid value
s.fillna(method='bfill') # backfill with next valid value<jupyter_output><empty_output><jupyter_text>http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.Series.fillna.html  Deal with duplicates<jupyter_code>s = pd.Series([6, 7, 7, 8, 1, 2, 3, 4, 5, 6, 7, 7, 8, 9, 10, 2, 3, 3, 6, 4, 7, 6, 5])
s
s.nunique() # number unique values (as above)
s = s.drop_duplicates() # drop duplicates removes duplicate values (keeping the first occurence and its index)\
s
s = s.sort_values()
s.reset_index()
# old index is added as an extra column by default, to avoid this, use `drop=True`
s.reset_index(drop=True)<jupyter_output><empty_output><jupyter_text>Aggregate Series<jupyter_code>s1 = pd.Series([1,2,3])
s2 = pd.Series([4,5,6])
pd.concat([s1, s2]) # note that the index is also concatenated
pd.concat([s1, s2], ignore_index=True) # regenerate a new index<jupyter_output><empty_output><jupyter_text>Value LookupGotcha to be aware of<jupyter_code>s = pd.Series([1,2,3], index=['a', 'b', 'c'])
3 in s # this checks if 3 is in the series INDEX
3 in s.index # equivalent to `3 in s`
3 in s.values # this checks if 3 is in the series VALUES<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1408705074950334, 'fraction_numerical': 0.02799349828426946, 'mean_word_length': 5.167037861915367, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '224466', 'license': 'MIT', 'n_tokens_mistral': 2101, 'n_tokens_neox': 1993, 'n_words': 694, 'path': 'Series.ipynb', 'repo_name': 'streamnsight/pandas-cheat-sheet', 'size': '5543'}