<jupyter_start><jupyter_text><jupyter_code>import numpy as np
np_load_old = np.load
np.load = lambda *a, **k: np_load_old(*a, allow_pickle=True, **k)
from tensorflow.keras.datasets import reuters

(X_train, y_train), (X_test, y_test) = reuters.load_data(num_words=None,
                                                         skip_top=0,
                                                         maxlen=None,
                                                         test_split=0.2,
                                                         seed=723812,
                                                         start_char=1,
                                                         oov_char=2,
                                                         index_from=3)
# Demo of encoding

word_index = reuters.get_word_index(path="reuters_word_index.json")

print(f"Iran is encoded as {word_index['iran']} in the data")
print(f"London is encoded as {word_index['london']} in the data")
print("Words are encoded as numbers in our dataset.")
len(word_index)
# TODO - your code!
from tensorflow.keras.preprocessing import sequence
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Embedding
from tensorflow.keras.layers import LSTM

max_features = 2000
# cut texts after this number of words (among top max_features most common words)
maxlen = 80
# taking the first 80 chars will make the results converge faster, though the model can take variable size inputs
batch_size = 32
# relatively small batch size for RNN

print('Pad sequences (samples x time)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
# This is a pre-processing technique that chops X into 80 char chunks and turn them into time sequences


print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)

print('Build model...')
model = Sequential()
model.add(Embedding(max_features, 128))
model.add(LSTM(128, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation='sigmoid'))

# try using different optimizers and different optimizer configs
model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

print('Train...')
model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=15,
          validation_data=(x_test, y_test))
score, acc = model.evaluate(x_test, y_test,
                            batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)<jupyter_output>WARNING: Logging before flag parsing goes to stderr.
W0823 17:23:02.268977 140125158094720 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0823 17:23:02.307673 140125158094720 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor<jupyter_text>Part 2- CNNs Find the FrogTime to play "find the frog!" Use Keras and ResNet50 (pre-trained) to detect which of the following images contain frogs:<jupyter_code>!pip install google_images_download
from google_images_download import google_images_download

response = google_images_download.googleimagesdownload()
arguments = {"keywords": "animal pond", "limit": 5, "print_urls": True}
absolute_image_paths = response.download(arguments)
from PIL import Image, ImageOps
import os
import numpy as np
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
from matplotlib.pyplot import imshow
# TODO - your code!

path = './downloads/animal pond/'

def process_img_path(img_path):
    return image.load_img(img_path, target_size=(224, 224))

model = ResNet50(weights='imagenet')

def get_img_results(img):
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)
    features = model.predict(x)
    results = decode_predictions(features, top=5)[0]
    return results

files = []
results = {}

for file in os.listdir(path):
    filename= path+file
    files.append(filename)
    img = process_img_path(filename)
    results[filename] = get_img_results(img)
    
for key in results.keys():
    files.append(key)
def display_image(path):
    pil_im = Image.open(path)
    imshow(np.asarray(pil_im));
for pred in results[files[0]]:
    print(str(pred[1]) + ': ' + str(round(pred[2]*100,2)) + '%')
display_image(files[0])
for pred in results[files[1]]:
    print(str(pred[1]) + ': ' + str(round(pred[2]*100,2)) + '%')
display_image(files[1])
for pred in results[files[2]]:
    print(str(pred[1]) + ': ' + str(round(pred[2]*100,2)) + '%')
display_image(files[2])
for pred in results[files[3]]:
    print(str(pred[1]) + ': ' + str(round(pred[2]*100,2)) + '%')
display_image(files[3])
for pred in results[files[4]]:
    print(str(pred[1]) + ': ' + str(round(pred[2]*100,2)) + '%')
display_image(files[4])<jupyter_output>jigsaw_puzzle: 86.8%
web_site: 6.41%
bib: 2.13%
comic_book: 1.57%
television: 0.54%<jupyter_text>Part 3 - AutoencodersDescribe a use case for an autoencoder given that an autoencoder tries to predict its own input. __*Your Answer:*__ Autoencoders use an unsupervised learning approach where the image becomes the target label. One use case could potentially be removing noise from sound or image files  Part 4 - More...Answer the following questions, with a target audience of a fellow Data Scientist:- What do you consider your strongest area, as a Data Scientist?Statistics and predictive modeling are my strong areas.- What area of Data Science would you most like to learn more about, and why?I would like to learn most about artificial neural networks and such because it is where the industry is leading.- Where do you think Data Science will be in 5 years?I believe data science will be at the forefront of tech companies and ideally general industries looking to keep up with current technologies. - What are the threats posed by AI to our society?Automation and the lack of job opportunities for humans as they have been replaced by machines.- How do you think we can counteract those threats? Developmental considerations allbeit potentially an inevitable threat.- Do you think achieving General Artifical Intelligence is ever possible?Yes, and I believe it is currently in existence across specific industries.A few sentences per answer is fine - only elaborate if time allows.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09394631639063393, 'fraction_numerical': 0.027841233580810965, 'mean_word_length': 3.824380165289256, 'pattern_counts': {'":': 3, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '366644', 'license': 'MIT', 'n_tokens_mistral': 2161, 'n_tokens_neox': 2017, 'n_words': 669, 'path': 'LastSprint.ipynb', 'repo_name': 'khaloodi/DS-Unit-4-Sprint-3-Deep-Learning', 'size': '7004'}