<jupyter_start><jupyter_text>Prerequesites  PythonYou first need to set up a Python environment (if you do not have done so already). The easiest way to do this is by installing Anaconda. We will be using Python 3, so be sure to install the right version.link: https://docs.anaconda.com/anaconda/user-guide/getting-started/ Why Python?Many data-heavy applications are now developed in Python- Highly readable, less complexity, fast prototyping- Easy to offload number crunching to underlying C/Fortran/...- Easy to install and import many rich libraries- numpy: efficient data structures- scipy: fast numerical recipes- matplotlib: high-quality graphs- scikit-learn: machine learning algorithms- tensorflow: neural networks Required packagesNext, you'll need to install several packages that we'll be using extensively. You'll need to run these commands on the command line. Installing packages with condaIf you are using Anaconda, you can use the conda package manager to install all packages:conda install numpy scipy scikit-learn matplotlib pandas pillow Installing packages with pipWith most other setups (not conda), you can use pip to install all packages. Pip is the Python Package index. It is included in most Python installations.pip install numpy scipy scikit-learn matplotlib pandas pillow<jupyter_code>!pip install numpy scipy scikit-learn matplotlib pandas<jupyter_output>Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (1.19.5)
Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (1.4.1)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.6/dist-packages (0.22.2.post1)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (3.2.2)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (1.1.5)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (1.0.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/[...]<jupyter_text>Virtual environmentsIf you are not using Anaconda, and you already have a custom Python environment set up, possibly using a different Python version, it may be wise to set up a virtual environment for this course so that it does not affect your existing environment: https://docs.python-guide.org/dev/virtualenvs/  TensorflowTo install TensorFlow 2, follow these instructions (https://www.tensorflow.org/install/) for your OS (Windows, Mac, Ubuntu). While installation with conda is possible, they recommend to install it with pip, even with an Anaconda setup.<jupyter_code># Requires the latest pip
!pip install --upgrade pip

# Current stable release for CPU and GPU
!pip install tensorflow<jupyter_output>Collecting pip
[?25l  Downloading https://files.pythonhosted.org/packages/54/eb/4a3642e971f404d69d4f6fa3885559d67562801b99d7592487f1ecc4e017/pip-20.3.3-py2.py3-none-any.whl (1.5MB)

[K     |▏                               | 10kB 16.2MB/s eta 0:00:01
[K     |▍                               | 20kB 21.7MB/s eta 0:00:01
[K     |▋                               | 30kB 12.6MB/s eta 0:00:01
[K     |▉                               | 40kB 11.1MB/s eta 0:00:01
[K     |█                               | 51kB 10.6MB/s eta 0:00:01
[K     |█▎                              | 61kB 10.5MB/s eta 0:00:01
[K     |█▌                              | 71kB 11.4MB/s eta 0:00:01
[K     |█▊                              | 81kB 11.3MB/s eta 0:00:01
[K     |██                              | 92kB 10.3MB/s eta 0:00:01
[K     |██▏                             | 102kB 10.9MB/s eta 0:00:01
[K     |██▍                             | 112kB 10.9MB/s eta 0:00:01
[K     |██▋                             | 122kB 10.9MB/[...]<jupyter_text>KerasKeras and TensorFlow have a shared history together. With TensorFlow 2.0, you should be using tf.keras rather than the separate Keras package.For more readings: https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/  Alternative environments for running the notebooks Google ColabGoogle Colab allows you to run a notebook on Google Drive (with limited GPU support): https://colab.research.google.com/notebooks/gpu.ipynb A more detailed tutorial can be found here (you won't need PyTorch for this course, but you can check it out): https://towardsdatascience.com/fast-ai-lesson-1-on-google-colab-free-gpu-d2af89f53604There are limitations (obviously): right now GPU usage is limited to 12h and RAM is shared among multiple users.Note: You need to upload your course notebooks to colab yourself (File > Upload Notebook). You can install additional packages from within notebooks with '!pip install package'.   Data representation for NN<jupyter_code>import numpy as np
#Scalar
x = np.array(12)
print(x.ndim)
#Vectors (1D tensors)
x = np.array([12, 3, 6, 14, 7])
x.ndim
#Matrices (2D tensors)
x = np.array([[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]])
x.ndim
#3D tensors and higher-dimensional tensors
x = np.array([[[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]],
[[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]],
[[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]]])
x.ndim
#Useful properties of ndarrays:
my_array = np.array([[1, 0, 3], [0, 1, 2]])
my_array.ndim     # number of dimensions (axes), also called the rank
my_array.shape    # a matrix with n rows and m columns has shape (n,m)
my_array.size     # the total number of elements of the array
my_array.dtype    # type of the elements in the array<jupyter_output><empty_output><jupyter_text>Indexing and SlicingArrays can be indexed and sliced using [start:stop:stepsize]. Defaults are [0:ndim:1]For multi-dimensional arrays, axes are comma-separated: [x,y,z].<jupyter_code>a = np.arange(10)**2
print(a)
print(a[3:10:2])
print(a[-1])
print(a[::-1])
b = np.arange(16).reshape(4,4)
print(b)
b[2,3] # row 2, column 3
b[0:3,1] # Values 0 to 3 in column 1 

b[ : ,1] # The whole column 1

b[1:3, : ] # Rows 1:3, all columns<jupyter_output><empty_output><jupyter_text>Tensor operations<jupyter_code>#Element-wise operations
import numpy as np
x = np.random.random((32, 10))
y = np.random.random((32, 10))
z = x + y
z = np.maximum(z, 0.)
#Brocasting
import numpy as np
x = np.random.random((64, 3, 32, 10))
y = np.random.random((32, 10))
z = np.maximum(x, y)
#Tensor dot
#An element-wise product is done with the * operator in Numpy, Keras, Theano,
#and TensorFlow. dot uses a different syntax in TensorFlow, but in both Numpy and
#Keras it’s done using the standard dot operator:
import numpy as np
x = np.random.random((1, 10))
y = np.random.random((10, 1))
z = np.dot(x, y)
#Shape manipulation
a = np.floor(10*np.random.random((3,4)))
a
a.transpose()
b = a.ravel() # flatten array
b
b.reshape(3, -1) # reshape in 3 rows (and as many columns as needed)<jupyter_output><empty_output><jupyter_text>Application - Logistic Regression for Breast Cancer data<jupyter_code># Load and prepare data from sklearn built-in datasets
from sklearn import datasets
from sklearn import model_selection
import numpy as np
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
import pandas as pd

#Load Wisconsin Breast cancer Dataset
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-wisconsin.data')
#Set column names
df.columns = ['id','clump_thickness','unif_cell_size','unif_cell_shape','marg_adhesion','single_epith_size','bare_nuclei','bland_chrom','norm_nucleoli','mitoses','class']
#View data
df.head()
#Drop IDs
df.drop(['id'], inplace=True, axis=1)
df.head()
#Replace missing values
df.replace('?', -99999, inplace=True)
# Set class IDs: Mapping class values to binary, it is 2 and 4 in our data. (2 for benign, 4 for malignant)
df['class'] = df['class'].map(lambda x: 1 if x == 4 else 0)
# Separate features and the target variable
X = np.array(df.drop(['class'], axis=1))
y = np.array(df['class'])
X.shape, y.shape
# Normalize the data
scaler = preprocessing.MinMaxScaler()
X = scaler.fit_transform(X)
#Split into training and testing
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2)
y_test
X_train.shape, y_train.shape, X_test.shape, y_test.shape
#Create model
import tensorflow as tf
model = tf.keras.models.Sequential([
    #tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(1,activation='sigmoid')
])
# Define optimizer and loss function
opt = tf.keras.optimizers.SGD()

model.compile(optimizer=opt,
loss='binary_crossentropy',
metrics=['accuracy'])
history = model.fit(X_train, y_train, epochs=100,batch_size = 64, validation_data=(X_test, y_test))
model.summary()<jupyter_output>Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 1)                 10        
=================================================================
Total params: 10
Trainable params: 10
Non-trainable params: 0
_________________________________________________________________<jupyter_text>Performance Evaluation<jupyter_code>import matplotlib.pyplot as plt
# summarize history for accuracy
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
## Find misclassified sample
print('Testing labels', y_test)


y_pred = (model.predict(X_test) > 0.5).astype("int32")
y_pred
y_pred = np.array([y_pred[i][0] for i in range(len(y_pred))])
print()
print('Predicted labels',y_pred)
y_test
idx_misc = (y_test != y_pred)
idx_misc
np.where(idx_misc)[0]
X_test[22,:]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13420575031044035, 'fraction_numerical': 0.04556309103066195, 'mean_word_length': 4.057971014492754, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 33, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 19, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '441770', 'license': 'MIT', 'n_tokens_mistral': 3865, 'n_tokens_neox': 3460, 'n_words': 1071, 'path': '[Intro_to_DL]Getting_started_Logistic_Regression_for_Breast_Cancer_data.ipynb', 'repo_name': 'FadouaKhm/CSE-590---Spring-2021---Intro-to-ML-and-DL-Tutorials', 'size': '10469'}