<jupyter_start><jupyter_text>Contents1&nbsp;&nbsp;Loop Like a Native [video]. <NAME>, 2013.1.1&nbsp;&nbsp;Generators2&nbsp;&nbsp;pd.MultiIndex3&nbsp;&nbsp;pd.DataFrame.groupby3.1&nbsp;&nbsp;Ex 4.13.2&nbsp;&nbsp;Ex 4.2<jupyter_code># standard imports
import numpy as np
import pandas as pd
import seaborn as sns; sns.set()
import matplotlib as mpl
import matplotlib.pyplot as plt

# embed static images in the ipynb
%matplotlib inline<jupyter_output><empty_output><jupyter_text>The `<f8` data type may be a bit confusing to you. First off, the `<` in the dtype indicates the [byte order](https://numpy.org/doc/stable/reference/generated/numpy.dtype.byteorder.html). `<` is the little-endian indicator. > On [Endianness](https://en.wikipedia.org/wiki/Endianness): In computing, endianness is the ordering or sequencing of bytes of a word of digital data in computer memory storage or during transmission. Endianness is primarily expressed as big-endian or little-endian. Big-endian systems store the most significant byte of a word at the smallest memory address and the least significant byte at the largest. A little-endian system, in contrast, stores the least-significant byte at the smallest address.`f8` indicates that the respective column has a float datatype. ----  Scratch Paper  Loop Like a Native [[video]](https://youtu.be/EnSu9hHGq5o). <NAME>, 2013.<jupyter_code><jupyter_output><empty_output><jupyter_text>Looping: `zip` vs. `enumerate`<jupyter_code>letters = ['a', 'b', 'c']
nums = [0, 1, 2]
# Goal: Create (letter, int) pairs

# Without zip
for idx, letter in enumerate(letters):
    num = nums[idx]
    print(f"{letter} {num}")

# With zip
for num, letter in zip(nums, letters):
    print(f"{letter} {num}")<jupyter_output>a 0
b 1
c 2
a 0
b 1
c 2<jupyter_text>What's the advantage of using zip? - We've got less code and we're not distracted by an unnecessary index when we only want to deal with the elements of the iterables. - Powerful: `zip` is a function that takes in iterables as values and produces a value which is itself an iterable. **Note** that the two collections have to have the same number of elements for `zip`. One advantage of using `enumerate` to keep an index is that the number of elements won't need to match for the two collections. Since `zip` accepts a stream of pairs, we can use it to generate dictionaries:<jupyter_code>letters = ['a', 'b', 'c']
nums = [0, 1, 2]
dict(zip(letters, nums))
import numpy as np 
buildings = ['b' + str(i) for i in np.arange(10)]
heights = np.random.random(10)
bh: dict = dict(zip(buildings, heights))

# Print the height of the tallest building in 'bh'
print(max(bh.values()))

# Print the building-height pair corresponding to the tallest building
print(max(bh.items(), key=lambda b: b[1])) # max of the seq sorted by seq[1] 

# Print the name of the tallest building
print(max(bh, key=bh.get))<jupyter_output>0.9906713395043445
('b6', 0.9906713395043445)
b6<jupyter_text>Generators<jupyter_code># Create a generator to return evens from an iterable.

def evens(seq):
    for n in seq:
        if n % 2 == 0:
            yield n<jupyter_output><empty_output><jupyter_text>Even if you use a generator on an infinite sequence, it will run without issue. This is becuase generators have what's known as "lazy" iteration. Generators look functions that have return statements but have important differences. Running a generator doesn't necessarily start at the beginning of its source code because `yield` preserves the current state of execution, while `return` ends a function. Generators create iterator objects, so the execution only ends if the generator reaches a return statement or it runs out of values to yield.  ----  Just Python<jupyter_code>x = {"apple", "banana", "cherry"}
y = {"google", "microsoft", "apple"}
x.difference_update(y)
x<jupyter_output><empty_output><jupyter_text>----  NumPy<jupyter_code>import numpy as np
# Generate 1D array of random ints b/w 0 and 5 with size 15.
np.random.randint(6, size=15)
# Generate a 2 by 4 matrix (np.ndarray) containing ints b/w 0 and 4 (inclusive).
np.random.randint(5, size=(2, 4))<jupyter_output><empty_output><jupyter_text>---- Pandas<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>cl: `pd.Series.transform` calls "func" to transform the values of a pd.Series object.cl: The first argument of `pd.DataFrame.transform` is a function or list of functions.Q: Given `df`, mimic the `df + 1` operation with `pd.DataFrame.transform`.<jupyter_code>df = pd.DataFrame({'A': range(3), 'B': range(1, 4)})
df
df + 1
df.transform(lambda x: x + 1)<jupyter_output><empty_output><jupyter_text>Q: Display the column types of `df`.<jupyter_code>df.dtypes<jupyter_output><empty_output><jupyter_text>Q: What type do string columns have in Pandas?<jupyter_code>df.astype(str).dtypes<jupyter_output><empty_output><jupyter_text>Q: Convert `df` to string type and add a "0" to the end of each element.<jupyter_code>df.astype(str).transform(lambda s: s + "0")<jupyter_output><empty_output><jupyter_text>Q: Return the square root and exponential of `s` in a pd.DataFrame.<jupyter_code>s = pd.Series(np.arange(1,5))
s
s.transform([np.sqrt, np.exp])<jupyter_output><empty_output><jupyter_text>--- `pd.MultiIndex` Q: What is the interface for hierarchical indexing in Pandas?<jupyter_code>arrays = [[1, 1, 2, 2],
          ['red', 'blue', 'red', 'blue']]
pd.MultiIndex.from_arrays(arrays, names=('number', 'color'))
# pd.MultiIndex is the interface for hierachcal index objects.
arrays = [['Falcon', 'Falcon', 'Parrot', 'Parrot'],
           ['Captive', 'Wild', 'Captive', 'Wild']]
index = pd.MultiIndex.from_arrays(arrays, names=('Animal', 'Type'))
df = pd.DataFrame({'Max Speed': [390., 350., 30., 20.]},
                   index=index)
df<jupyter_output><empty_output><jupyter_text>--- `pd.DataFrame.groupby`<jupyter_code>df = pd.DataFrame({
    'Animal': ['Falcon', 'Falcon', 'Parrot', 'Parrot'],
    'Max Speed': [380., 370., 24., 26.]})
df
df.groupby(['Animal']).mean()<jupyter_output><empty_output><jupyter_text>We can groupby different levels of a hierarchical indexusing the `level` parameter:<jupyter_code>arrays = [['Falcon', 'Falcon', 'Parrot', 'Parrot'],
           ['Captive', 'Wild', 'Captive', 'Wild']]
index = pd.MultiIndex.from_arrays(arrays, names=('Animal', 'Type'))
df = pd.DataFrame({'Max Speed': [390., 350., 30., 20.]},
                   index=index)
df
df.groupby(level=0).mean()
df.groupby(level="Type").mean()<jupyter_output><empty_output><jupyter_text>Q: What does `pd.Series.value_counts` do?Return a Series containing counts of unique values. Q: Return the number of non-null values in `s`.<jupyter_code>s = pd.Series([0.0, 1.0, np.nan])
s.count()
pd.DataFrame(s).notnull().sum()
import pandas as pd
from typing import Iterable

def frequency_encode(X: pd.DataFrame, cat_features: Iterable[str]):
    """Frequency encode categorical features of X

    Args:
        X (pd.DataFrame): feature matrix
        cat_features (Iterable[str]): list of categorical feature names
    """
    for feature in cat_features:
        assert isinstance(feature, str), "Column names must be strings."
        assert feature in X.columns, "Feature '{feature}' is not in X.columns"
        # Swap out each category for the frequency count of the category
        X[feature] = X[feature].groupby(X[feature]).transform('count')
# Make pd.DataFrame from sklearn's Boston dataset.
from sklearn.datasets import load_boston
import pandas as pd
boston = load_boston()
# Set column indices as feature names
boston_df  = pd.DataFrame(boston.data, 
                          columns=boston.feature_names,) 
boston_df['PRICE'] = boston.target # Specify 'PRICE' as the target variable
boston_df.head()<jupyter_output><empty_output><jupyter_text>Q: Return a pd.Series containing the counts of unique values in the 'RAD' column.<jupyter_code>boston_df['RAD'].value_counts()<jupyter_output><empty_output><jupyter_text>---- 4. Data Visualization - Matplotlib & Seaborn<jupyter_code># standard imports
import numpy as np
import pandas as pd
import seaborn as sns; sns.set()
import matplotlib as mpl
import matplotlib.pyplot as plt

# embed static images in the ipynb
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Q: What is the function of `%matplotlib inline` in a Jupyter notebook?- Plots will be embedded in the notebook as static images.Q: What is the function of `%matplotlib notebook` in a Jupyter notebook?- Plots will be embedded in the notebook as interactive plots.Q: How can you embed interactive plots in a Jupyter notebook?- ```%matplotlib notebook```Q: How can you embed static impages of plots in a Jupyter notebook?- ```%matplotlib inline```  Ex 4.1<jupyter_code># Generate simple random walk plot 
rng = np.random.RandomState(1)
x = np.linspace(0, 10, 500)
y = np.cumsum(rng.randn(500, 6), 0)
plt.plot(x, y)<jupyter_output><empty_output><jupyter_text>Q: What does the `np.cumsum` method do?> Return the **cumulative sum** of the elements of a `np.ndarray` along a given axis.Q: What does the `np.linspace` method do?> Return a `np.ndarray` of evenly spaced numbers over a specified interval.  Ex 4.2<jupyter_code># plot sine and cosine graphs from -pi to pi
x = np.linspace(start=-np.pi, stop=np.pi, num=100)  
fig = plt.figure()
plt.plot(x, np.sin(x), '-')
plt.plot(x, np.cos(x), '--')<jupyter_output><empty_output><jupyter_text>Q: What is the type of `plt.figure`?  : `function` | a method inside the matplotlib.pyplot moduleQ: What does the `plt.figure` method do?- Create a new figure, or activate an existing figure.Q: The figure object generated by `plt.figure()` has what type?- `mpl.figure.Figure`<jupyter_code>fig = plt.figure()
type(fig)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12452520275125757, 'fraction_numerical': 0.016938712657838004, 'mean_word_length': 5.028465346534653, 'pattern_counts': {'":': 0, '<': 92, '<?xml version=': 0, '>': 92, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '123104', 'license': 'MIT', 'n_tokens_mistral': 3386, 'n_tokens_neox': 3205, 'n_words': 1076, 'path': 'notebooks/Anki Notebook - Cookbook Generator.ipynb', 'repo_name': 'Unique-Divine/LeetCode-Coding-Practice', 'size': '9757'}